2_Bag of Words.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Competition - Spooky Author Identification\n",
    "**Identify horror authors from their writings**\n",
    "\n",
    "**File descriptions**\n",
    "\n",
    "    train.csv - the training set\n",
    "    test.csv - the test set\n",
    "    sample_submission.csv - a sample submission file in the correct format\n",
    "\n",
    "**Data fields**\n",
    "\n",
    "    id - a unique identifier for each sentence\n",
    "    text - some text written by one of the authors\n",
    "    author - the author of the sentence (EAP: Edgar Allan Poe, HPL: HP Lovecraft; MWS: Mary Wollstonecraft Shelley)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Bag of Words"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## This notebook inlcudes the Bag of Words approach for feature extraction, bechmarking of different classifers, preparing submissions to Kaggle"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## References\n",
    "http://scikit-learn.org/stable/tutorial/text_analytics/working_with_text_data.html\n",
    "\n",
    "http://scikit-learn.org/stable/auto_examples/text/document_classification_20newsgroups.html#sphx-glr-auto-examples-text-document-classification-20newsgroups-py\n",
    "\n",
    "http://scikit-learn.org/stable/modules/feature_extraction.html#text-feature-extraction\n",
    "\n",
    "http://scikit-learn.org/dev/auto_examples/model_selection/grid_search_text_feature_extraction.html\n",
    "\n",
    "http://scikit-learn.org/stable/modules/feature_extraction.html#applications-and-examples\n",
    "\n",
    "http://scikit-learn.org/stable/modules/ensemble.html#using-the-votingclassifier-with-gridsearch\n",
    "\n",
    "https://www.kaggle.com/marcospinaci/talking-plots-1-sklearn-classifiers-0-334"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Environment Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 170,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Common import\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import os\n",
    "import itertools\n",
    "\n",
    "# Set graph parameters\n",
    "%matplotlib inline\n",
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "plt.rcParams['axes.labelsize'] = 14\n",
    "plt.rcParams['xtick.labelsize'] = 12\n",
    "plt.rcParams['ytick.labelsize'] = 12\n",
    "\n",
    "# Set path to save visualizations\n",
    "PROJECT_DIR = \".\"\n",
    "IMAGES_DIR = \"images\"\n",
    "\n",
    "# Function to save visualizations\n",
    "def save_fig(fig_id, tight_layout=True):\n",
    "    path = os.path.join(PROJECT_DIR, IMAGES_DIR, fig_id + \".png\")\n",
    "    print(path)\n",
    "    print(\"Saving figure\", fig_id)\n",
    "    if tight_layout:\n",
    "        plt.tight_layout()\n",
    "    plt.savefig(path, format='png', dpi=300)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load the dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 171,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set the path of the spooky author dataset\n",
    "SPOOKY_PATH = 'spooky'\n",
    "\n",
    "\n",
    "def load_spooky_dataset(dataset_type):\n",
    "    filepath = os.path.join(SPOOKY_PATH, dataset_type, dataset_type + \".csv\")\n",
    "    print(filepath)\n",
    "    return pd.read_csv(filepath)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 172,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "spooky\\train\\train.csv\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>text</th>\n",
       "      <th>author</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>id26305</td>\n",
       "      <td>This process, however, afforded me no means of...</td>\n",
       "      <td>EAP</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>id17569</td>\n",
       "      <td>It never once occurred to me that the fumbling...</td>\n",
       "      <td>HPL</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>id11008</td>\n",
       "      <td>In his left hand was a gold snuff box, from wh...</td>\n",
       "      <td>EAP</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>id27763</td>\n",
       "      <td>How lovely is spring As we looked from Windsor...</td>\n",
       "      <td>MWS</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>id12958</td>\n",
       "      <td>Finding nothing else, not even gold, the Super...</td>\n",
       "      <td>HPL</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        id                                               text author\n",
       "0  id26305  This process, however, afforded me no means of...    EAP\n",
       "1  id17569  It never once occurred to me that the fumbling...    HPL\n",
       "2  id11008  In his left hand was a gold snuff box, from wh...    EAP\n",
       "3  id27763  How lovely is spring As we looked from Windsor...    MWS\n",
       "4  id12958  Finding nothing else, not even gold, the Super...    HPL"
      ]
     },
     "execution_count": 172,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train = load_spooky_dataset(\"train\")\n",
    "train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 173,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "spooky\\test\\test.csv\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>id02310</td>\n",
       "      <td>Still, as I urged our leaving Ireland with suc...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>id24541</td>\n",
       "      <td>If a fire wanted fanning, it could readily be ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>id00134</td>\n",
       "      <td>And when they had broken down the frail door t...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>id27757</td>\n",
       "      <td>While I was thinking how I should possibly man...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>id04081</td>\n",
       "      <td>I am not sure to what limit his knowledge may ...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        id                                               text\n",
       "0  id02310  Still, as I urged our leaving Ireland with suc...\n",
       "1  id24541  If a fire wanted fanning, it could readily be ...\n",
       "2  id00134  And when they had broken down the frail door t...\n",
       "3  id27757  While I was thinking how I should possibly man...\n",
       "4  id04081  I am not sure to what limit his knowledge may ..."
      ]
     },
     "execution_count": 173,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test = load_spooky_dataset(\"test\")\n",
    "test.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 174,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 19579 entries, 0 to 19578\n",
      "Data columns (total 3 columns):\n",
      "id        19579 non-null object\n",
      "text      19579 non-null object\n",
      "author    19579 non-null object\n",
      "dtypes: object(3)\n",
      "memory usage: 459.0+ KB\n"
     ]
    }
   ],
   "source": [
    "train.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 175,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 8392 entries, 0 to 8391\n",
      "Data columns (total 2 columns):\n",
      "id      8392 non-null object\n",
      "text    8392 non-null object\n",
      "dtypes: object(2)\n",
      "memory usage: 131.2+ KB\n"
     ]
    }
   ],
   "source": [
    "test.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Split dataset for training and testing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 301,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(train['text'], train['author'])\n",
    "#X_train, y_train = train['text'], train['author']\n",
    "#X_test = test['text']\n",
    "#X_ID = test['id']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 302,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(14684,) (4895,) (14684,) (4895,)\n"
     ]
    }
   ],
   "source": [
    "print(X_train.shape, X_test.shape, y_train.shape, y_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Extracting features from text - Bag of Words approach"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Tokenizing Text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 303,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((14684, 22082), (14684,))"
      ]
     },
     "execution_count": 303,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "\n",
    "count_vect = CountVectorizer(stop_words='english')\n",
    "X_train_counts = count_vect.fit_transform(X_train)\n",
    "X_train_counts.shape, X_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 304,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  (0, 5694)\t1\n",
      "  (0, 15111)\t1\n",
      "  (0, 11295)\t1\n",
      "  (0, 7101)\t1\n",
      "  (0, 19565)\t1\n",
      "  (0, 4775)\t1\n",
      "  (0, 4778)\t1\n"
     ]
    }
   ],
   "source": [
    "print(X_train_counts[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Occurences to frequencies - Term Frequency times Inverse Document Frequency"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 305,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(14684, 22082)"
      ]
     },
     "execution_count": 305,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.feature_extraction.text import  TfidfTransformer\n",
    "tf_transformer = TfidfTransformer()\n",
    "X_train_tfidf = tf_transformer.fit_transform(X_train_counts)\n",
    "X_train_tfidf.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 306,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  (0, 4778)\t0.251038917494\n",
      "  (0, 4775)\t0.393671188645\n",
      "  (0, 19565)\t0.255879097696\n",
      "  (0, 7101)\t0.456917074663\n",
      "  (0, 11295)\t0.281321257897\n",
      "  (0, 15111)\t0.382529390885\n",
      "  (0, 5694)\t0.531304758441\n"
     ]
    }
   ],
   "source": [
    "print(X_train_tfidf[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train a classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 307,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "MultinomialNB(alpha=1.0, class_prior=None, fit_prior=True)"
      ]
     },
     "execution_count": 307,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Mutlinomial variant of Naive Bayes provides a good baseline for word counts\n",
    "from sklearn.naive_bayes import MultinomialNB\n",
    "\n",
    "clf = MultinomialNB()\n",
    "clf.fit(X_train_tfidf, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Evaluation of performance on test set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 308,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((4895,), (4895, 22082))"
      ]
     },
     "execution_count": 308,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Prediction on test set\n",
    "X_test_counts = count_vect.transform(X_test)\n",
    "X_test_tfidf = tf_transformer.transform(X_test_counts)\n",
    "X_test.shape, X_test_tfidf.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 309,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = clf.predict(X_test_tfidf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 310,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy Score: 0.811235955056\n",
      "Accuracy: 0.811235955056\n"
     ]
    }
   ],
   "source": [
    "# Accuracy\n",
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "print(\"Accuracy Score:\", accuracy_score(y_test, y_pred))\n",
    "print(\"Accuracy:\", np.mean(y_pred == y_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 311,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Log Loss: 0.604572239607\n"
     ]
    }
   ],
   "source": [
    "# Log loss\n",
    "from sklearn.metrics import log_loss\n",
    "\n",
    "y_pred_prob = clf.predict_proba(X_test_tfidf)\n",
    "\n",
    "print(\"Log Loss:\", log_loss(y_test, y_pred_prob))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 312,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Baseline submission\n",
    "#create_submission_file(y_pred_prob, 'MNB_Baseline')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 313,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             precision    recall  f1-score   support\n",
      "\n",
      "        EAP       0.75      0.88      0.81      1964\n",
      "        HPL       0.90      0.72      0.80      1434\n",
      "        MWS       0.83      0.81      0.82      1497\n",
      "\n",
      "avg / total       0.82      0.81      0.81      4895\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Detailed report\n",
    "from sklearn.metrics import classification_report, confusion_matrix\n",
    "\n",
    "print(classification_report(y_test, y_pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 314,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1735,   82,  147],\n",
       "       [ 308, 1027,   99],\n",
       "       [ 257,   31, 1209]], dtype=int64)"
      ]
     },
     "execution_count": 314,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Confusion Matrix\n",
    "conf_mx = confusion_matrix(y_test, y_pred)\n",
    "conf_mx"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 315,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Plot confusion matrix\n",
    "# http://scikit-learn.org/stable/auto_examples/model_selection/plot_confusion_matrix.html#sphx-glr-auto-examples-model-selection-plot-confusion-matrix-py\n",
    "def plot_conf_matrix(cm, classes, title='Confusion Matrix', cmap=plt.cm.Blues):\n",
    "    plt.imshow(cm, interpolation='nearest', cmap=cmap)\n",
    "    plt.title(title)\n",
    "    plt.colorbar()\n",
    "    tick_marks = np.arange(len(classes))\n",
    "    plt.xticks(tick_marks, classes, rotation=45)\n",
    "    plt.yticks(tick_marks, classes)\n",
    "    fmt = 'd'\n",
    "    thresh = cm.max()/2\n",
    "    for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):\n",
    "        plt.text(j, i, format(cm[i, j], fmt),\n",
    "                 horizontalalignment=\"center\",\n",
    "                 color=\"white\" if cm[i, j] > thresh else \"black\")\n",
    "    plt.tight_layout()\n",
    "    plt.ylabel('True Label')\n",
    "    plt.xlabel('Predicted Label')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 316,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV4AAAEqCAYAAABdmDjNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xd8VFX6x/HPl9CbgIgCCigWEBRU\nrGBDVGwriq6uFVBxbWt33Z8FLNjdtYsdy65rQxRFVBTFgrKggoCgIqBU6RBKCMnz++PcwBBThmQy\nM5k8b1/3lZl7bnnuBJ+cOffcc2RmOOecS55qqQ7AOeeqGk+8zjmXZJ54nXMuyTzxOudcknnidc65\nJPPE65xzSeaJtxKTZJJ2LqF8iqTDkhhSQkm6SNJCSdmSti7HcbIl7ZTI2JJN0pmSPkh1HC4xPPGm\ngKRZktZLalpo/XdRMm1ThmMOkXR77Doz62Bmn0TlA6NjnxqzT/XY80XHWB8lqlWSJkg6tJTz7irp\nNUmLJa2QNEnSVZKytvQaCh23BvBP4Cgzq29mS8p6rGj/X8oTT1ES8XuU1CbatnpJ25nZv83sqPJF\n7NKFJ97UmQn8peCNpD2AOhV8zqXAraUkxXvMrD6wFfA4MLS47SW1Bb4GfgP2MLOtgFOBLkCDcsa6\nLVAbmFLO41S0Cv89lpaUXeXjiTd1XgTOiXl/LvBC7AaSPpF0fsz7PpI+L3wgSf2BM4Hrotrq8Gj9\nLEk9YjYdCawHziotODPLB/4DNCEkwaLcAnxpZleZ2fxov+lmdoaZLY9i+FPU5LE8up72MXHPknRN\nVEteIekVSbUl7QpMjzZbLunjomqGsZ+PpJ0lfRodZ7GkV2K229gkI2krSS9IWiRptqQbJVWL/Xwl\n3SdpmaSZko4p5aOK5/d4nKRvJa2U9JukgTHFY2KuM1vSgVEcX0j6l6SlwMDY372kg6Jr3CF63yn6\nfNuVEqtLE554U+croKGk9lGN8jTgpbIcyMyeBP5NVFs1sxOK2xS4CRgQfZUvVhTTOYQa3cJiNusB\nvF7CMXYFXgauALYBRgDDJdWM2ezPQE9gR2BPoI+Z/Qh0iMobmVn3kmKN3AZ8ADQGtgceLma7hwm1\n+Z2AQwnX2DemfH9C0m8K3AM8I0klnDee3+Pq6DyNgOOAiyT1isoOiX42in53Y2Pi+AVoBgyKPZiZ\nfQk8ATwvqQ4h+d9oZtNKiNOlEU+8qVVQWzoSmAbMregTmtnbwCLg/GI2uUbSckKyeAC4yczyitl2\na2B+Cac7DXjXzD40s1zgPsLX8INitnnIzOaZ2VJgONA5/qvZTC7QGmhhZuvMrKhvBgWJ8R9mtsrM\nZgH3A2fHbDbbzJ6Krvl5oDnF1/gLlPh7NLNPzOx7M8s3s0mEP0Yltp0D88zsYTPbYGZriygfSPgD\nMg6YBzxayvFcGvHEm1ovAmcAfSj09bSC3QjcQGhDLew+M2tESJBdgHtL+Lq9hJCYitMCmF3wJmq+\n+A1oGbPNgpjXa4D6pUZftOsAAeOipo1+RWzTFKgZG1P0ush4zGxN9LK0mEr8PUraX9LoqHljBfDX\nKJaS/FZSYfSHbAjQEbjffLSrSsUTbwqZ2WzCV/ljgaFFbLIaqBvzfruSDrcF5/0Q+Bm4uIRtzMwm\nA18Qvh4XZRTQu4RTzSPUQgGIvrLvQNlq9qujn0V+Hma2wMwuMLMWwIXAY/pjV7vFbKoZF2hVxng2\niuP3+B/gbWCH6AbkYMIfCSj+91bi71NSS2AA8Bxwv6RaZQjdpYgn3tQ7D+huZquLKPsOOFlS3SiJ\nnFfCcRYS2i3jdQOhllis6GZNN4rvWTAAOEjSvZK2i/bZWdJLkhoBrwLHSToialO+GsgBvtyCOAEw\ns0WEBHmWpKyoRts2JtZTJW0fvV1GSFx5hY6RF8U0SFIDSa2Bqyhj23ohJf0eGwBLzWydpP0IteMC\ni4B8tuB3F/0BGwI8E513PqGN21USnnhTzMxmmNn4Yor/ReiFsJDQ3vjvEg71DLB7dHd7WBzn/YLQ\nPlhYQc+I1YSbVc8RbuQUGTtwINAGmBJ9jX4DGA+sMrPphB4UDxNqmycAJ5jZ+tLiK8YFwLWEJo4O\nbJ7A9wW+lpRNqF1ebmYzizjGZYTa8y/A54Ta6LNljGejUn6PFxO68a0CbiYk/4L91hBunn0R/e4O\niON0fyO0O98UNTH0BfpKOrhcF+GSRt405JxzyeU1XuecSzJPvM45l2SeeJ1zLsk88TrnXJJV2cE3\nVL2OqWZ5x3HJTJ3at0p1CGnLb0YX77fZs1myZHFJj1dvsayGrc02FPXg3uZs7aL3zaxnIs9dkapu\n4q3ZgFq7/TnVYaSlT794KNUhpK11ucU9Pe2OOjSennBbxjasjev/03XfPVrak4BppcomXudcZSBQ\n5rWIeuJ1zqUvAdXKNaZ+WvLE65xLbyWOylk5eeJ1zqUxb2pwzrnk8xqvc84lkfAar3POJZf85ppz\nziWdNzU451wy+c0155xLLuE1XuecSzqv8TrnXDIJsjLv5lrm/SlxzmWOgu5kpS3xHEq6VNJ4STmS\nhhQqqyvpMUmLJa2QNCamTJLulrQkWu6JJhwtKO8saYKkNdHPzqXF4onXOZfepNKX+MwDbqfoyU2f\nBJoA7aOfV8aU9Qd6AZ2APYHjgQtDaKoJvEWYqboxYVLat6L1xfLE65xLY0pYjdfMhprZMMIs1ZvO\nIO0G/Anob2aLzCzPzCbEbHIucL+ZzTGzucD9QJ+o7DBCk+0DZpZjZg+FoOleUiyeeJ1z6S2+Gm/T\nqBmhYOm/BWfYH5gN3BI1NXwvqXdMeQdgYsz7idG6grJJtvkI+ZNiyovkN9ecc+lLcT+5ttjMupTx\nLNsDHYE3gBbAgcC7kqaa2Q9AfWBFzPYrgPpRO2/hsoLyEqe38Rqvcy69JaipoQRrgVzgdjNbb2af\nAqOBo6LybKBhzPYNgeyollu4rKB8VUkn9MTrnEtvibu5VpxJpZRPIdxYK9ApWldQtmdsLwfCDbgp\nlMATr3MujSXu5pqk6pJqA1lAlqTakqoDY4BfgX9E23Ql3DR7P9r1BeAqSS0ltQCuBoZEZZ8AecDf\nJNWSdGm0/uOSYvHE65xLb4mr8d5IaFa4Hjgren2jmeUCJwLHEtpnnwLOMbNp0X5PAMOB74HJwLvR\nOsxsPaGr2TnAcqAf0CtaXyy/ueacS18SVEtMmjKzgcDAYsqmEG6qFVVmwHXRUlT5t8A+WxKLJ17n\nXHrzQXKccy7JfJAc55xLMq/xOudcEskHQnfOuaRTNU+8Lk6DB5zJMYd0ZNHSVXQ59Q4AXryrL7u0\n2RaARg3qsHzVWg44/S66dGjNIzf9BQh/4AcNHsHbo0Of7mnv3sKq1Tnk5eezIS+fbmfek5oLSpJH\nH3qAF4Y8gyR279CRx558lkv/ej7ffjOBGjVqsE+XfXngkcHUqFEj1aFWuMsvvoAPR46g6TbbMObr\n7zYre+yhf3LLjdczdeY8tt66KY8+eD9vvPoyABs2bOCn6dOY+ss8GjdpkorQEyZMQOFNDS5OLw7/\nisGvfMrTt52zcd3Z1z+38fVdV53Eiuy1AEyZMY+uZ95DXl4+2zVtyNev/IN3x0wmLy8fgJ79H2TJ\n8tXJvYAUmDd3LoMfe5hx306mTp06nHvmabzx2n/58+ln8NRzLwJw3rln8vxzT3N+/4tSHG3FO/3M\ncziv/8VcemHfzdbPnfMbn378Edvv0Grjuksuv5pLLr8agPffe4cnHn2o0iddIMq8qQ4i8TKvDp8m\nvvhmBktXrCm2vPeRe/PqyDDy3Np1uRuTbK2aNdh8oKOqJW/DBtauXcuGDRtYu3YN2zVvwVE9j0US\nktiny37Mmzs31WEmxYFdD6ZR48Z/WH/zP67h5tvuKLYm+OZrr3DSKadVdHhJoo2/+5KWysYTbwp0\n3bstC5euYsavizau27djaya8fgPjX/s//jbovxsTsZkx/LFL+eLf19Hv5K6pCjkpWrRsyWVXXE3H\nXduw644tadhwK47ocdTG8tzcXP778kv0OPLoFEaZWiNHDGe75i3psEenIsvXrFnD6FEfcPyfTkpy\nZBXHE28FkTRL0lpJ2THLIzHlh0kySdcV2q9NtL5gn1mSrk/+FWyZP/fswmsjx2+27n+TZ7PPKYPo\ndtY9XNvvKGrVDK1A3fv+i4POuJtelz7GhacdTNe926Yi5KRYtmwZ777zNpN+mMH0X+awZvVqXnn5\npY3lV11+CV27HsxB3Q5OYZSps2bNGh649y7+fsOAYrf54L132PeAAzOjmSFSrVq1UpfKJp0iPsHM\n6scsl8aUnQssjX4WpZGZ1Qf+AtwsqWdFB1tWWVnVOLF7J15//5siy6fPXMjqtevpsHMLAOYvCkN9\nLlqWzdsfT2LfDm2SFWrSffLxKFq3aUPTbbahRo0anNDrJL7+aiwAdw26lSWLFnHHPfenOMrUmTVz\nBr/OnkX3rl3o0nEX5s2dw5EH78/vCxds3GbYG69mUDMDm9p4S1sqmXRKvEWSVBc4BbgE2EVSsYMd\nm9lYwnBsHZMU3hbrvv9u/DhrIXN/X75xXesWW5OVFX4VrZo3Ztc22zJ73hLq1q5J/bq1AKhbuyY9\nDmzHlBnzUhJ3MuywQyvGj/uaNWvWYGZ8OvpjdtutPc8/9zQfffgBz7zwn0pZu0mU3TvswdRf5jJ+\n8k+Mn/wTLVpuz4effU2zbbcDYOWKFYz9/DN6HvenFEeaOMrQNt7K0KuhN2Gw4deAMwmjAI0vvFE0\nHuZBhCk3vi3qQNF0IGFKkBr1KybayPN39uHgfXahaaP6/DzyNm4bPILnh43l1KP32XhTrcBBe+3E\nNX2PIndDHvn5xuV3vMKS5atp03JrXvnnBQBUz8rilffG8+GXP1Ro3KnUZb/9OfGk3hxyYBeqV6/O\nnp060+e8C2i+dQN2aNWaIw8LbdwnnHgSf/+/m1IcbcW7sO9ZfPn5GJYuWUzndjty7f/dzJnn9C12\n+xHvvMWh3XtQr169JEZZ8SpjYi2N0uEOuqRZQFNgQ8zqa83sKUmjgMlmdoWkvwAPAS3MLFdSG2Am\nYSg3AxYAj0cTzpWoWt1mVmu3Pyf2QjLEwrGlfnxV1rrcvFSHkLaOOvQAvvtmQkKzZPWtd7KGx95e\n6nbLXjpzQjmm/km6dKrx9jKzUbErJO0AHA78I1r1FmEa5uOAYTGbNjWz2KTtnMsEAlXLvBpvujeY\nnU2IcbikBcAvQG1Cc4NzrgpIVBuvpEujGYhzJA0pZpsBUU+pHjHrakl6VtJKSQskXVVonyMkTZO0\nRtJoSa1LiyXdE+85wC1A55ilN3CcpK1TGZhzruIl+ObaPOB24NkizyW1JdzIn1+oaCCwC9Ca8A38\nuoKeU5KaAkOBm4AmhPtPr5QWSDol3uGF+vGOBtoAj5rZgpjlbeBnQtcx51yGS1TiNbOhZjYMWFLM\nJo8AfwcKT9tzDnCbmS2Lpnt/CugTlZ0MTDGz18xsHSFJd5LUrqRY0qKN18zabOH2HWLeZl4DkHNu\nk/j+D28qKba305Nm9mTcp5BOBdab2YjYRC6pMdACmBiz+UTCPGsQelFtLDOz1ZJmROunUYy0SLzO\nOVckEW/f7cVl7dUgqT5wB3BUEcUF/U5XxKxbATSIKV/E5mLLi+SJ1zmX1pLQj/cW4EUzm1lEWXb0\nsyGwLub1qpjyhoX2iS0vUjq18Trn3GaS9OTaEcDfoh4LC4AdgFcl/d3MlhFutsWOStSJ8IQs0c+N\nZZLqAW1jyovkidc5l94SNFaDpOqSagNZQJak2pKqExJvRzb1nJoHXAg8Gu36AnCjpMbRTbMLgCFR\n2ZtAR0m9o2PfDEwys2Lbd8ETr3MunSmhw0LeCKwFrgfOil7faGZLYntOAXnAMjMraGYYAMwAZgOf\nAvea2UgAM1tE6OI6CFgG7A+cXlog3sbrnEtriRoYycwGErp7lbZdm0Lvc4B+0VLU9qOAEruPFeaJ\n1zmX3jKww6gnXudcWsvE0ck88Trn0lZlHW+3NJ54nXNpzROvc84lWSYOC+mJ1zmX1rzG65xzySRP\nvM45l1QCMjDveuJ1zqUz79XgnHNJV81vrjnnXBLJmxqccy6phNd4nXMu6bzG65xzSeY315xzLomk\nzGxq8IHQnXNpLHFT/0i6VNJ4STmShsSsP0DSh5KWSlok6TVJzWPKJeluSUui5R7FnFRSZ0kTJK2J\nfnYuLRZPvM65tCaVvsRpHnA78Gyh9Y2BJ4E2QGvCRJXPxZT3J0zn3gnYEzieMDUQkmoCbwEvRcd5\nHngrWl8sT7zOubSWqBqvmQ01s2HAkkLr3zOz18xspZmtAR4BusZsci5wv5nNMbO5wP1An6jsMEKT\n7QNmlmNmDxE6Y3QvKRZPvM659BVHbTfKu02jZoSCpX85znoIm88S3AGYGPN+YrSuoGySmVlM+aSY\n8iL5zTXnXNragn68i82sS7nPJ+1JmCn4xJjV9YEVMe9XAPWjdt7CZQXlDUo6jyde51xaS1Z3Mkk7\nA+8Bl5vZZzFF2UDDmPcNgWwzM0mFywrKV5V0Lm9qcM6ltQTeXCvhHGoNjAJuM7MXCxVPIdxYK9CJ\nTU0RU4A9tflfhz3ZvKniD6psjXf3Xbbn9RF3pzqMtHTOS9+kOoS0NeSMvVIdQtpSRUwHnMDxeCVV\nJ+S8LCBLUm1gA7At8DHwqJkNLmLXF4CrJI0ADLgaeDgq+wTIA/4maTBwQbT+45JiKTbxSqob7wVF\ndwKdcy6hEjwe743AgJj3ZwG3EJLpTsAASRvLzax+9PKJqPz76P3T0TrMbL2kXtG6u4AfgF5mtr6k\nQEqq8WZHAcUjK87tnHNuCyhhT66Z2UBgYDHFt5SwnwHXRUtR5d8C+2xJLCUl3mO25EDOOVcRqtRY\nDWb2fjIDcc65P8jQ8Xjj7tUgaevoWed/Sdo6WrevpB0qLjznXFUW2ngT8+RaOomrV4OkTsBHwCKg\nLeGO3hLgT4Tnm8+uoPicc1VcZUyspYm3xns/8LSZtQdyYta/Bxyc8Kiccy5SrZpKXSqbePvx7ks0\nGk8hc4HtEheOc87FyNA23ngTbw5/fCwOYFdgceLCcc65TZSh07vH29QwHLgxevIDwCS1JHQYfrNC\nInPOOZLzyHCyxZt4rwG2BxYAdQiPw/0CrAf+r2JCc845qCaVulQ2cTU1mNkySQcCPYG9CQn7G2CE\nmeVXYHzOuSosU+dci3uQnCjBjogW55xLigzMu1v0AEVPSR9ImhMtH0ryx4qdcxUqEx+giCvxSroM\neBtYCNwRLQuAYZL+VnHhOeequky8uRZvU8P1wJVm9mjMusckjSUMtfZQwiNzzlV5ooLG+U2xeJsa\ntgKKGjTnA4ru3+ucc+UnkVWt9KWy2ZJ+vCcWsf5E/Gabc64CJaqpIRrka7ykHElDCpUdIWmapDWS\nRkdTARWU1ZL0rKSVkhZIuirefYtT0gwUF8e8/Z7wAMXBwFfRugOAQ4F7Sr1i55wrA0Ei++nOA24H\njiY8jxDOITUFhgLnEyqZtwGvEHIchMHTdwFaE4ZIGC1pqpmNjGPfIpXUxntToffrgP2jJXbdZcCd\nJZ3EOefKKlF518yGhuOpC+GBsAInA1PM7LWofCCwWFI7M5sGnAP0NbNlwDJJTwF9gJFx7FukkgZC\nb17mK3TOuQSJs7tYU0njY94/aWZPxnmKDsDEgjdmtlrSDKCDpIVAi9jy6HWv0vYFtjzxOudcqknE\ne/NssZl1KeNp6hPGGo+1AmgQlRW8L1xW2r7FijvxSmpDqFa3AmrGlpnZxUXs4pxz5ZaEPgvZ/LF3\nVkNgVVRW8H5dobLS9i1WvDNQHEl4gGIam6rWOxFmFx4XzzGcc64skvBk2hTg3Jjz1SPMtDMlGqdm\nPtAJ+DDapFO0T4n7lnTCeLuT3QHcZWZ7EcbmPY1Q8/2UkJCdcy7hQq+G0pe4jiVVl1SbUGHMklQ7\nGur2TaCjpN5R+c3ApJibYy8QenU1ltQOuAAYEpWVtm+R4k287YCXotcbgDpmtjo6ybVxHsM557ZM\nHOM0bEGN+EZgLeFJ3LOi1zea2SKgNzAIWEbouXV6zH4DgBnAbEJl814zGwkQx75FireNdzWb2nXn\ns6kqbcDWcR7DOee2WKKGhTSzgYQ+uUWVjSJUMIsqywH6RcsW7VuceBPvOOAg4AdC37V7JbUnZHpv\n43XOVYiCpoZME2/ivZZN3SoGAI2B84AfCQ9QuBLkrFvH2Scfzfr1OWzYsIGjj+vFZdfeyJxfZ3H1\nRX1YvnwZu3fsxN0PP03NmjWZN+c3/nFFf1atWEFefh5X/d+tHHrE0am+jIS59OA2dGm1FSvWbuDy\noeEeRP1aWVzTvS3N6tfk9+z13PvRDFavz+OQtk04uVOYT3Vdbj6Dv5jNrKVrabFVLa7t3nbjMbdt\nUIuXJ8xl+JTfU3JNyTD40Yd4/rlnAOOcPudx0aWX8/2kiVx9+cVkZ6+mVevWPPnsizRsmFnDp1TG\nYR9LE1cbr5lNN7MJ0etVZtbXzHYFTiIpvT0qt5q1avHca+8ybNRXvPnhWD7/ZBTfTRjH/YNu4pwL\nLuH9LyayVaNGvPHy8wAMfvBuep5wMkM//JL7Hx/Crf+4MsVXkFgf/7SYW0f+tNm63p2aM2nuSi5+\nbTKT5q6kd5RsF67K4YZ3pnPF0Km8+u08Lu4WHoOftyKHK9+cypVvTuXqYVPJ2ZDPV7OXJ/1akmXq\nlMk8/9wzfDRmLJ999Q3vv/cuM37+icsvuZABt97Bl//7juNP6MXDD9yX6lATTnEslU3cA6EXY3fg\np1K3quIkUa9e+MKwITeX3NxcJPHV559y9PEnAXDiqWfy0ch3Nm6fvSp0A1y1ciXNts2shwinLsgm\nO2fDZuv2a9WI0T8tAWD0T0vYv3VjAKb/vprV6/M2vt66Xk0K27NFQxasymFR9voKjjx1fpw+jX33\n25+6detSvXp1uh58CO+8PYyff5rOQd0OAeCwI3ow/K3MmntWysw518qbeF2c8vLyOKnHgXTbc0cO\nOqQ7rVrvSMOtGlG9emjt2a55SxYumAfAJVffwPCh/+WwfXblr2f35sZBmVeLKaxRneosW5sLwLK1\nuWxV54+tYD12a8o3c1b8YX23nZrw2YwlFR5jKrXfvQNffvEZS5csYc2aNXz4/nvMnTuHdrt34L13\nhwPw1tDXmTvntxRHmnjVqqnUpbJJi8QraZakHoXW9ZH0eUz5WknZkhZKek5S/ajsE0nnpyLuLZGV\nlcWbo8YyesJ0vv9uPDN+nv6HbQraskYMe42T/nwWn0z4kcEvvsHfLzuf/PyqPadox+YN6LFbU14Y\nN2ez9dWrif1ab8UXM5elKLLk2K1dey6/6lpOOqEnp/Q6lg57dKJ6VhaPPP40Tz/xGId13Y/s7FXU\nqPnHbwSVXSbOQJEWiTdOJ5hZfcIsx/sS+uRVOg23asR+Bx7MxAn/Y+WK5WzYEL5yL5g/d2OTwusv\nP0/PE04GYK8u+5OTk8OypYtTFnMyLF+7gcZ1agDQuE4NVqzd1BTRukkdLj24NXd+8DOrcvI222/v\n7bfil8VrNts+U519bj8+/fJ/jPjgExo3bsxOO+/Crru1Y+jwkXzyxTh6n3o6O+64U6rDTChRejND\nxjU1SNq9pIXQnzepzGwu8B7QMdnnLqulSxaxckW48bNu7VrGfjaanXbZjf27HsL774Q2ubde+zfd\njz4OgBYtd+Crzz8BYMZP08jJWUeTrbdJSezJMu7X5Ry+S+gSfvguWzPu1/B5Na1Xk+uPaMu/PpnJ\nvJU5f9jv4LZNGDNjaVJjTZVFv4ceG7/99ivvvD2MU049feO6/Px87rv7Dvqed2EqQ0y8OGq7lTDv\nltqdbDLhIYniqJTyhJO0A3AsYfDhLd23P9AfQnJLlkULF/KPy/uTl59Hfn4+PU84mcOPPIadd23H\n1Rf14aF7bqN9xz055S/hke/rBtzBzddcxvNPPYIQd/7riYzqUnPV4TvSsXkDGtauztN/2ZP/TpjH\n0InzubZ7W3rs1pTF2eu55+MZAJy2d3Ma1K7OX7uG3gx5+cY1b/0AQM2sanRq2ZDHP5+dsmtJpnPO\nPJVlS5dSvXoN7v3nQzRq3JjBjz7E008+DsDxf+rFmef0SW2QFSCT/u0XkFnxeVPSbvEcxMz+2GC5\nJUFIs4CmhMeRC9QEvjGzboXKVwDvAleb2VpJnwAvmdnTW3LOjp32ttdHflaesDPW9e/+kOoQ0taQ\nM/ZKdQhp6/Bu+/PtN+MTmiW33bmjnXbf66Vu9/BJ7SeUY1jIpCuxxlvehLqFekWP3gHh5hphOo0i\ny51zVUMl7LRQKh8I3TmX1jzxpq+C4d4K5JlZbsqicc4lRLh5lnmZtzJ1JyvJ44Qh3gqW51IbjnMu\nURI1Hm86SYsar5m1KWLdEKLBhosqj9nusIqJyjmXaiLuOdcqlS2q8UqqL6mTpBoVFZBzzsWqFscS\nD0ltJI2QtEzSAkmPRDNQIKmzpAmS1kQ/O8fsJ0l3S1oSLfeonO0fccUsqZ6kF4CVwARgh2j9I5Ju\nKE8AzjlXkgQ+QPEY8DvQHOgMHApcLKkm8BZhlp3GwPPAW9F6CH3/exHmWtsTOB4o15Mq8f6xuJMw\nwvpBbJppE+AD4NTyBOCcc8VRHI8Lb8EjwzsCr5rZOjNbQJjUoQNwGKHZ9QEzyzGzhwitHN2j/c4F\n7jezOdGTs/cDfcpzXfEm3hOBy8zsKzZ/Um0qYbZh55yrEHHWeJtKGh+z9C/iUA8Cp0uqK6klcAyb\nku8k2/xpsknRetg0s3qBiTFlZRLvzbVtCFX0wuqV5+TOOVcSEUagi8PiOJ5c+5QwQ/BKwkzDzwPD\nCANuFR5vdAXQIHpdv1D5CqC+JFlJj/6WIN4a7wTC+AgFCk7WDxhblhM751w8EtHGK6ka8D5hjJd6\nhCEIGgN3A9lA4fmSGgKroteFyxsC2WVNuhB/4r0BuFvSw4Ra8iWS3iM0Ot9U1pM751yJ4ujDG2dv\nsyaETgGPRO24Swj9/Y8lzJi+Z6GeCntG64l+doop6xRTVibxzrk2hnAHsBkwFziZMOV7VzPzWYad\ncxVGcfxXGjNbDMwELpJUXVLwDaOoAAAblUlEQVQjwk2zicAnQB7wN0m1JF0a7fZx9PMF4CpJLSW1\nAK4mesagrOJ+gCKa7PK08pzMOee2RIKndz8ZeAD4OyHRjgauNLP1knoBTwN3AT8QBuUqmMTvCUIn\ngu+j909H68osrsQrqW5J5Wa2pjxBOOdccRL15JqZfUfoOlZU2bfAPsWUGXBdtCREvDXebEoe8Dwr\nAbE459xmElzjTRvxJt5jCr2vAexFGC/Xb6455ypGJZ3apzRxJV4ze7+I1e9I+hE4i9D47JxzCVcZ\nJ7MsTXmHhRzPpsfqnHMuoQqaGnxYyEg0gMQlhO5lzjlXAURWBtZ44+3VsIjNb64JaASsB86pgLic\ncw5Rhdt4Cc8yx8oHFgFfmllRYzg451z5VdKmhNKUmnijgYJzgRHRUGrOOZc0VfLmmpltAB4BalV8\nOM45t0lBU0OCBkJPG/E2NYwjDAwxuwJjcc65P8jEOdfiTbyPAPdHA0RMIAyQs5GZTU10YM45JzJn\nKvRY8SbeV6Ofj0U/C3o4KHrtjww75xJPYfqfTBNv4m1foVE451wxMi/tlpJ4JT0LXG5m05MUj3PO\nbRSeXMu81Fta88m5QJ1kBOKcc0XJxEeGS0u8lfCSnHOZQ0ilL3EfTTpd0g+SVkuaIengaP0RkqZJ\nWiNptKTWMfvUkvSspJWSFki6qrxXFc8NwzJP6Oacc+VR0KuhtCWuY0lHEia37EuYQfgQ4BdJTQmT\nYN5EmJttPPBKzK4DgV2A1sDhwHWSepbnuuK5ubagtL8oZua9GpxzFSKBvRpuAW41s6+i93Oj4/cH\nppjZa9H7gcBiSe3MbBphPJq+ZrYMWCbpKaAPMLKsgcSTePsDy8t6AuecK484025TSeNj3j9pZk9u\nPIaUBXQB3pb0M1AbGAZcC3QgTHoJgJmtljQD6CBpIdAitjx63atMFxOJJ/EOz8SBcCSoWT0Tu2aX\n33/7dEl1CGnrwEEfl75RFTV9waqEH1Mi3mEhF5tZSf9wtyXMnHMKcDBh/Jm3CAOA1ScM+hVrBaE5\non7M+8JlZVZa5vH2XedcSiXo5tra6OfDZjY/mu79n8CxhDklGxbaviGwKiqjUHlBWZl5rwbnXFpT\nHEtpovbZORRdmZxCGIsmnE+qB7QltPsuA+bHlkevp2zxhcQoMfGaWbVMbGZwzlUeCRyd7DngMknN\nJDUGrgDeAd4EOkrqLak2cDMwKbqxBmFOyRslNZbUDrgAGFKea/JGTudc2grdyVTqEqfbgP8BPwI/\nAN8Cg8xsEdAbGAQsA/YHTo/ZbwAwgzA646fAvWZW5h4NUI4515xzruIpYY8Mm1kucHG0FC4bBbQr\nZr8coF+0JIQnXudcWsvAoRo88Trn0ldBU0Om8cTrnEtflXRqn9J44nXOpTVPvM45l0Qi7ifXKhVP\nvM65tCZv43XOueTKwAqvJ17nXHrzGq9zziVRmHMt1VEknide51z6UuKeXEsnnnidc2kt89KuJ17n\nXBrL1OndPfE659Ja5qVdT7zOuXSXgZnXE69zLq1lYlODD4TunEtriZj6Z7PjSbtIWifppZh1Z0ia\nLWm1pGGSmsSUNZH0ZlQ2W9IZ5b0mT7zOufSW6MwLjxJmogiHlzoATwBnE2YjXgM8Vmj79VHZmcDj\n0T5l5k0Nzrm0FfJq4poaJJ0OLAe+BHaOVp8JDDezMdE2NwE/SGoA5BOmBepoZtnA55LeJiTp68sa\nh9d4nXPpK46JLqMm4KaSxscs/f9wKKkhcCtwdaGiDsDEgjdmNoNQw901WvLM7MeY7SdG+5SZ13id\nc2ktzntri82sSynb3AY8Y2a/afOD1gdWFNp2BdAAyCuhrMw88Trn0pgS0tQgqTPQA9iriOJsoGGh\ndQ2BVYSmhuLKyswTr3MurSWoN9lhQBvg16i2Wx/IkrQ7MBLotOl82gmoRZgGPh+oLmkXM/sp2qQT\nMKU8wXjiTYJ5c+dwzaXns/j3hVSrVo3Tzu5H3/6X8OA9t/PKS8/RZOumAFx9wy0c3qMnb73+X556\n9F8b9582dTJvj/qS3ffoVNwpMsK6devocfghrM/JYUPeBk46+RRuGnALjz/6CI88/AC/zJjBb/MX\n0bRp01SHWmEG/Kkdh+zalKWr13Pq4+MAuOLIthyya1Ny84w5S9cy4K0fyM7ZAEC/bq05ca/m5Ocb\n94z8ibEzlgLwl/235+S9WyBg6Dfz+M/Xc1J1SeVStk4LRXoS+G/M+2sIifgioBkwVtLBwDeEduCh\nZrYKQNJQ4FZJ5wOdgROBg8oTjCfeJKhePYv/u+VOOu65F9nZqzixR1e6HdodgL4XXsYFl1yx2fYn\nnnI6J55yOgDTp07mwnP+nPFJF6BWrVqM/PBj6tevT25uLt0P7cZRRx/DgQd15djjjueoHoelOsQK\nN/y7Bbwybg63nbT7xnVfzVjGw6N+Ic+Mv/VoS7+DW/PQqBns1LQuR3doximPfc02DWox+Oy96PXI\nWHZsWo+T927B2U+NJzfPePSsTnz+0xJ+Xbo2hVdWDgnIvGa2htBNLBxSygbWmdkiYJGkvwL/BrYG\nRgF9Y3a/GHgW+B1YAlxkZl7jTXfNtm1Os22bA1C/fgN23nU3Fs6fF9e+w998leNPPrUiw0sbkqhf\nvz4Aubm5bMjNRRKd9yqqWS4zffPrcppvVXuzdV/9snTj6+/nrKBH+2YAHNZuG96f8ju5eca85ev4\nbekaOrZsSLOGtfh+zkrWbcgHYMLs5Rzebhue//LX5F1IAlXEk2tmNrDQ+/8A/ylm26VAr0Se37uT\nJdmcX2cz5fuJdNpnXwBefHYwxx66H3+//EJWLF/2h+3fHfYGJ5z052SHmTJ5eXnsv09nWrVoRvce\nR7Lf/vunOqS0cmLnFnzx8xIAtmlQiwUr1m0s+31VDs0a1GLG76vZu3UjtqpTndrVq9Ft563Zbqta\nqQq53BL//ETqJS3xSpolab2kpoXWfyfJJB0oaaWkrJiyp4pZNzh63UHSB5KWSVouaYKkY5N1TVtq\ndXY2F/f7Czfddg8NGjTkzD4XMHrcFN4Z/RXbbLsddwzYvD/2dxPGUbtuXXZrX64ug5VKVlYWX0/4\njp9nzWH8/8YxZfLkVIeUNs47uDV5+caI7xcCRd90MmDm4jUM+WI2j5+9F4+e1ZkfF2azId+SG2yi\nxJN1K2HmTXaNdybwl4I3kvYA6kRvFwNZwN4x2x8MzCu07hBgTPR6OPAh4VG+ZsDfgJUVEXh55ebm\nckm/Mzix9+kcfXz41tK02bZkZWVRrVo1Tj+rHxO/nbDZPu8Me50TTqoazQyFNWrUiEMOPYwPPhiZ\n6lDSwgmdtuOQXZpyw9BNTYu/r8xhu5hmiWYNarFoVQ4Aw76dzxlP/o/zhnzDirW5/LqkkrbvUtCh\nrOT/KptkJ94XgXNi3p8LvBC9zgW+IiRWJDUDagKvFFq3KzAmqjnvCDxlZuuj5Qsz+zwpV7IFzIzr\nr7iItrvuxnkX/W3j+t8Xzt/4+oMRb7Nru003VPLz83nv7aEc36vqJN5FixaxfPlyANauXcvHH41i\nt93apTiq1DuobRP6dG3NFf+dtLHdFuCT6Ys5ukMzamSJFo1q02rrukyeG+odjevWAGC7hrXo3n4b\nRk5emJLYy0vE/eRapZLsm2tfAWdLak/oI3ca0A24PSofQ0iy90c/P4+WS2PWzTSzOQqd8X4GXpL0\nNDDWzEr81xU9RtgfoMX2OyT40oo34euxDHvtP+zWviPHHx7aLK++4RbeGfoaU6dMQojtW7Xi9vse\n3rjPuLGfs12LlrRqs2PS4ky1BfPnc0G/c8nLyyPf8ul9yp859rjjefThh/jn/fewcMEC9t17T3r2\nPJbHn3w61eFWiDtP7sA+bRrRqG4NRl55EIM/mUnfbq2pmVWNx8/uDMD3c1Yy6N3p/LJoNR9M/Z03\nLj6AvPx87hoxnYIWhfv+vAeN6tZgQ14+d434kVXrNqTwqsqnMibW0sgsOW0/kmYB5wMHAPWATwnP\nTB9DqO3uSOhX9wbQFHgAmAy8DMwAtovWNTSzvtExtycMVHFMtP/nwHkxHZ2LtUfnve2tD79I2PVl\nkhaN65S+URV14KCPUx1C2pr+xEWsmTc9oWmyY6e97fWRpX+Jbd+i3oQ4HhlOG6no1fAicAbQh03N\nDAW+IjxR0pFQu/0sGhHot5h1Be27mNkcM7vUzNoCrYHVRRzTOVeJZWJTQ9ITr5nNJtxkOxYYWqhs\nHWGczOOB5mY2LSr6LFq3JzGJt9C+vxHGzexYMZE751IhAzs1pKwf73lAdzNbXUTZGOAKwniZBT6P\n1i2IhmxDUmNJt0jaWVK16GZbP0Kt2TmXKTIw86Yk8ZrZDDMbX0zxp4SuYbENO59H62Jru+sJbcKj\nCF3IJgM5hCYM51wGkMKTa6UtlU3SejWYWZti1m8g5m+Wmb1Pob9hUW+FwutWE7qjOecyWOVLq6Xz\nsRqcc+ktAzOvJ17nXBqrnE+mlcYTr3MurVXCJtxSeeJ1zqWtgkeGM40PC+mcS2uJGCRHUi1Jz0ia\nLWmVpG8lHRNTfoSkaZLWSBotqXWhfZ+NRkpcIOmq8l6TJ17nXFpL0JNr1QlPwB4KbAXcBLwqqU30\nDMDQaF0TYDxhcK4CA4FdCE/HHg5cJ6lnea7Jmxqcc2ktES0NUffTgTGr3pE0E9iHMN3PFDN7DUDS\nQGCxpHbR07PnAH3NbBmwTNJThOcFyjxmqdd4nXPpS2FKqNKWLT6stC1hiNkpQAdgYkFZlKRnAB0k\nNQZaxJZHr8s1O4HXeJ1zaWsLbq41lRT7NOyTZvZkkceUahAmtnzezKZJqg8sKrTZCqABYdCugveF\ny8rME69zLq3FWZ9dHM+wkJKqEUZIXE8Y5xsgG2hYaNOGwKqorOD9ukJlZeZNDc65tJaoYSGjyROe\nIUwV1tvMcqOiKUCnmO3qAW0J7b7LgPmx5dHrck3v7onXOZfWEjjn2uNAe+AEM4udhO5NoKOk3pJq\nAzcDk2KGpX0BuDEaEbEdcAEwpDzX5InXOZfWElHjjfrlXgh0BhZIyo6WM81sEdAbGAQsA/YHTo/Z\nfQDhZttswuiJ95pZuWZh9TZe51zaStQME9EEDMUeycxGAUXOrGpmOYSxvvuVP5LAE69zLq35IDnO\nOZdsmZd3PfE659JbBuZdT7zOuXRWOaf2KY0nXudc2vJhIZ1zziWE13idc2ktE2u8nnidc2nNu5M5\n51wSSVAt8/KuJ17nXJrzxOucc8nlTQ3OOZdkfnPNOeeSLAPzride51x6K8ucaunOE69zLm1l6pNr\nMrNUx5ASkhYRBjZOB02BxakOIk35Z1OydPp8WpvZNok8oKSRhGsszWIz65nIc1ekKpt404mk8fFM\n1FcV+WdTMv98Kicfq8E555LME69zziWZJ9708GSqA0hj/tmUzD+fSsjbeJ1zLsm8xuucc0nmidc5\n55LME69zziWZJ940pEx8RtI5t5En3jRkfsfTlZGkEyS1S3UcrmQ+VkMakXQBsB3wNTDezJamOCRX\niUh6BdgGGCRpppnlpDomVzTvTpYmJA0DWgLZQDPgQTN7UpK8BuxKEyXdFsBxwFozyy1UXs3M8lMS\nnPsDb2pIA5KGA3XMbF8zOxz4FLhQUvWqnnQlnSKpQ6rjSGeSugCtgJ5mttLMciVtI+lQSUcCmFm+\nJP//PU14U0OKSTqcUEs5LGb1W4Tabz1gRQrCSguSOgK3AW9LesbMfkx1TGnKgAZmtlpSM2Bv4AVg\nGdBY0hgzO8VrvOnD/wKmmJmNBgYA70vqLKkG8BTwsZlV2aQLYGaTgXOAA4DzJO1aUCYpK+b1VikI\nL+UkXSmpn5lNAGpLmgQ8B/wLeBg4FDgD2K+g5uvSg9d4U6ig3c3Mbou+Bo4H1gL/Z2YPR9tUuTZe\nSXcAbYFFwAPA5YREYpKeM7PpZpYXbXs50EnSX81sfcqCTjJJ9YD9CZWnZwm13NuBicBMM/s42m4N\nYdzpVSkK1RXBb66lgKR9gB/NbFWh9ZcBDwIdzWyqpBqFb5JkOklvEQa+/gjoTEgsvQi1twHAWOBZ\nM5su6RrgHmAfM/s2RSGnjKS2wATgXjMbVMw2FwJXAD3MbG4y43PF8xpvkkm6Hfg/4HNJg4BJZjYf\nwMwelrQN8J2ko81sdFW6Gy3pdaBlwcDeknoBdwHNzewjSQYMBJZHbZl/BfatSklX0l+Bp4FqZjZD\n0kXAQEljC2q50XbdgK7AtcBRnnTTiyfe5JsMzASWAkOAbyV9DDwC5JjZzZLygI8kHWJmn6cu1OSR\n1AQ4GnhQUl0zWwNMBdYA9aMml4+jh/oeItx87Ba1b1YJkk4GHiO0eY+SNAZ4jfC5HSNpgpmtiJoh\njom2O9zMvk9Z0K5I3tSQJJJqm9k6STsBo4FrgG+As4A+hPbM9whfG7MlXQp8ZGY/pCrmZJF0M6EJ\nYQ7wNuEP0nvAMOARM7un0PZdCHNszUpupKklaQfCZ7KakHCPIdRotyf84e5jZl9E29YhdFH0h3DS\nkPdqSAJJTwFnApjZL8B9hLbJfDO7hZCE9wFOAeZLugp4vIok3cZAHWAQ0AQ4EegHfAn8uyDpxvZB\nNbPxVS3pApjZb8BlQBbwC/AqMByoCcwDHpXUINp2rSfd9OWJt4JFN4v2MrNnYlaPBhYCrSQdQuj+\nczGwB6H9d1jBXftMZ2bLgEcJfZf/CTQGjgLmEtpym0TbVYl27sIk3SRprKSdoyaYL4F3gN3NbAhw\nKdCDMNPwnoR/Ry7NeVNDBZL0JrCNmXWLWVfTzNZLegw4m/CI8A1m9myq4kyF6Km8DTHvawB3AMcD\nfYGVhNrcy4THpxelJNAUkvR3wlOMg4AawCjgccJ4HncBV5rZj5JaAbsQupP1NbNpKQrZxckTbwWR\n9DThpkebmD6nOxFuDF0K5AAfAmPM7OKq1F9X0p2Em4yvA+vNzCS1ILR5TwYaAFcTanFjCTXhQVWp\n1hu18T8EfAD0B04AjgDaA6cTHiw5CDi0oP9yVeoBU9l5U0PFeZFwR/4oAEnbA2OA76L2yeXAt4Ta\nS1UbCnI0cDNwQpR0dyCMyPYg4Y/VMEIbeHPCQwIvV8GE8i7wGbAfoWnhXUL3uXcJyXgpoVnmrwXj\nN1fBz6jS8sRbQczsU0JN5Z9R38uxwGNmdmNUu11LSDaHS2pUlQY/N7MPgAuA26LP5ktgsJndGX07\n+A/wMXAj8JuZ/Zy6aJMvehx6HiHJnkv4Az0GqGdm1xC+MbUHdgVOBmqnKFRXRt7UUMGim2cjgHfM\n7PSY9VcSvi6ebmbTUxVfKhX12RR8XY6aHnLMbElKg0wiSQcC42KapvoCVxJqvU8QbqJ1M7OZkloS\n+ulOrqr/fiozT7xJIKkr4WmjG8xsaFTLu5PwGGeVeQCgKNFn8xRwA/B+9OBElSNpCOEP8dOEHi8D\noj9AdwHzzexBhTGb9yQ8iValvgVkGk+8SSLpUMJAL98RhoE80sy+SW1U6SH6bB4j3JV/oyoNdlNA\n0kGEttsXgB0BAe8T2rlrmtkV0XYfEu4L7BXbK8RVLp54kygae/c/hAGrJ6Y6nnQiqQfhW0D3woMH\nVRVR7X8wcBVhYPMGhHbuJsDRZvZhtN32ZjYnZYG6cvPEm2QFjw6nOo50FDNGQ5UV/XF+CLjZzN6U\n1BnYmfAI9VrvuZAZPPE6l2YkHUZo9/6Hmb2e4nBcBfDRyZxLM2b2iaTzgcejJ/qqZLt3JvMar3Np\nytu9M5cnXufSmLd7ZyZPvM45l2T+yLBzziWZJ17nnEsyT7zOOZdknnidcy7JPPE651ySeeJ1ZSZp\nsqSBMe9nSbomBXF0kWSS2lTgOQZKmpyA46TkM3LpxRNvBpE0JEpAJilX0i+S7pNUL0kh7EsYZaxU\nkvpIyq7geGLP94mkR5J1PudK4o8MZ55RhEk0awAHE8Z3rQdcVNTGkmqYWW4iTlwVJ6R0riy8xpt5\ncsxsgZn9Zmb/Af4N9IIw+EpUGz5W0jhJ6wlznCHpBEkTJK2TNFPSIEk1Cw4qqZmktyStlTRbUr/C\nJy78NVpSQ0mPS5ofHfcHSadFg8A8B9SLqaEPjPapKeluSXMkrZb0P0lHFzpPT0nTomN+RpgCp1wk\n3SVpenR9syTdI+kPU+pIOl/Sr9F2wyQ1LVTeV9LUKLYfJV0pyf8/c5vxGm/mW0uo/ca6mzCL78/A\nqiix/Ru4nDC3VyvCuLC1gIJEOgRoTZh+Zg3wL6BNcSeN5pB7jzAhY1/gR2A3wvxgXwJXEKZzbxvt\nUtDs8Fy07gxgDnAsMFzSvmY2MZoYcxhh9K5HCTMy/DP+j6NYq4F+wFxgd8L15wA3xWzTBjgLOBGo\nCzwJPAv8KbrmC4BbgcuACUDHKM5cwJs53CZm5kuGLITk+E7M+/0IU6S/Er0/DDCgd6H9xgA3FVrX\ni5AMRahRGtA1prw1kAcMjFk3C7gmen0kkA+0LybWPkB2oXVto31aFVo/jDBRKIRk/SPR4+7Ruhuj\n+NqU8Nl8AjyyBZ/lX4GfY94PjK63Vcy6btF5d4ne/wqcXeg4VwBTi/qMfKm6i9d4M0/P6KZVdUJN\n9y1CDSzW+ELv9wH2k/T3mHXVgDqEaWbaExLiuIJCM5staV4JcexFmCvshy2IfW9Cop9aaNLlWoRZ\nh4li+crMYgcZGbsF5yiSpFMISXJnoD6QFS2x5prZrzHvvyb64yJpObAD8ISkx2O2qU64Juc28sSb\necYQppXPBeZZ0TfOVhd6Xw24BXitiG0XUbbEUZZ9qhFqkPsS4o+1thzHLZGkA4D/Ej6DK4HlhOaD\n+7bgMAXtuAXT1TtXLE+8mWeNbfkMtN8A7YrbT9IPhMSyL1FSkdQKaFHKMZtLal9MrXc9f6xRfktI\nrNuZ2ehijjsV6C1JMbXeA0qIIx5dCbXZ2wpWSGpdxHYtJe1gZr9F7/cjfC4/mNlCSXOBtmb2Qjnj\ncRnOE6+DcEPoHUmzgVeBDYQbQ/uZ2XVmNl3SSMLX6P6E2uc/2VQLLcpHhK/ib0i6ktAuuzNQz8yG\nEdo6a0s6kpBw15jZj5L+DQyRdDUheTchtE3/YmZDCTe9rgYekPQYsAehlhmPptEcZrF+j2JrKelM\nQrPF0cBfith/LfC8pKsIzTCDgXfN7KeofCDwcNTsMILQ1LM30NLM7owzRlcFeDcXh5m9T5hy/nBC\nO+444HrCzaICfYCZhLbW4YTZkmeVcMx84BjgC+Al4AfgQaBmVP4lIXG9TGjOuC7atS+hZ8M9wDTg\nHeAQYHa036/AyUBPYCKhaeD6OC/1NEKSj12uMrPhwL3AA8Akwo3Bm4vYfxahSWJ49Dn8EsVbcM1P\nE3pGnB3F9hmh2WdmnPG5KsIHQnfOuSTzGq9zziWZJ17nnEsyT7zOOZdknnidcy7JPPE651ySeeJ1\nzrkk88TrnHNJ5onXOeeS7P8BHwqHMdzbgNwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2a39f888dd8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_conf_matrix(conf_mx, train.author.unique(), title=\"MultiNB Confusion Matrix\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Building a pipeline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 194,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.pipeline import Pipeline\n",
    "\n",
    "# Pipeline features - vectorizer -> transformer -> classifier\n",
    "# CountVectorizer - Text Preprocessing, tokenizing, and filtering of stopwords - dictionary of feature indices\n",
    "# TfidfTransformer - Term frequencies, and downscaling weights for words - tf and tf-idf\n",
    "# MultinomialNB - Naive Bayes Classifier - multinomial variant classifier\n",
    "\n",
    "mnb_clf = Pipeline([\n",
    "    ('vect', CountVectorizer()),\n",
    "    ('tfidf', TfidfTransformer()),\n",
    "    ('clf', MultinomialNB()),\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 195,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Pipeline(memory=None,\n",
       "     steps=[('vect', CountVectorizer(analyzer='word', binary=False, decode_error='strict',\n",
       "        dtype=<class 'numpy.int64'>, encoding='utf-8', input='content',\n",
       "        lowercase=True, max_df=1.0, max_features=None, min_df=1,\n",
       "        ngram_range=(1, 1), preprocessor=None, stop_words=None,\n",
       "        strip...inear_tf=False, use_idf=True)), ('clf', MultinomialNB(alpha=1.0, class_prior=None, fit_prior=True))])"
      ]
     },
     "execution_count": 195,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mnb_clf.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 196,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy score: 0.802655771195\n",
      "Log Loss: 0.60978207366\n"
     ]
    }
   ],
   "source": [
    "y_pred = mnb_clf.predict(X_test)\n",
    "y_pred_prob = mnb_clf.predict_proba(X_test)\n",
    "print(\"Accuracy score:\", accuracy_score(y_test, y_pred))\n",
    "print(\"Log Loss:\", log_loss(y_test, y_pred_prob))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 197,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV4AAAEqCAYAAABdmDjNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XeYVEXWx/HvbxiSBEExgIK4JhAF\nVMyYMLsGFHfVVRFR0de05nV3UXHFHNaEOSCmXSMK5oQYcBUUVEyogICCZBlyOO8fVQ1NO6Fnpqe7\n6TkfnvvQfeuG0z0zp6vr1q2SmeGccy57inIdgHPO1TaeeJ1zLss88TrnXJZ54nXOuSzzxOucc1nm\nidc557LME28ekGSSNi+nfKykvbMYUvK5j5f0ei7OXR2StpL0maR5ks6txnHukXRZJmPLBUklkv6Q\n6zhc4Im3GiRNkLREUouU9aNjMm1bhWMOlNQ/eZ2ZdTCzYbG8Xzz2n5L2KU4+XzzGkvjHNk/SKEl7\nlXPOfpKWxu3nSPpQ0q7x3I+b2QGVfR3pkFQvnnucpPnx/XyoKu9bKS4BhplZEzO7vaoHMbMzzOyq\nDMSzmqSf47kp68+L6/uleZxhkk6taDsza2xmP1YxXJdhnnirbzxwXOKJpG2BhjV8zlnAvyTVKWeb\nG8ysMbA2cDfwXAXb/zduvx7wftxeGYu4dM8AhwN/iXF2AkYB+2bg2JsAYzNwnJr0HXBSyrqecX1G\nSCrO1LFc5njirb5HCX8sCScBg5I3SK2VSOol6f3UA0nqAxwPXBJrn0Pi+gmS9kva9FVgCXBCRcGZ\n2QrgCWAdYIM0tl8KPAJsCKybGmuilibpR0kzJN0oqSipvLekryXNlvSapE1KO098PfsDR5jZJ2a2\nzMzmmtkAM3swbtNK0ouSZkn6XtJpSfv3k/SUpEGxVj9WUpdY9jawD3BnfB+3LO9noODfkn6VNFfS\n55K2iWWrfQORdFqMZVaMrVXKe3NGrMHPljSggg+vT4C1JHWI+3cgfGh/knTM5pKGSpoejzlU0sax\n7Gpgj6TXeWdSHGdJGgeMS1q3efyWMVrSOXF9HUkfSLq8nDhdhnnirb6PgKaS2sca5THAY1U5kJnd\nBzxOrK2a2WFlbQpcBlwhqW55x4wx9STUzKdVFIOk+kAvYLKZzShjsyOBLsD2wBFA77hvd+AfwFGE\nmvN7wJNlHGM/4GMzm1ROOE8Ck4FWwNHANZKSa8OHA/8BmgEvAncCmFm3eO6z4/tYUQ3yAGBPYMt4\nrGOAmakbSeoGXAv8GWgJTIznT3YosCOh9v5n4MAKzp38wf27D23C3+jDhBp8G2Bh0uv8Z8rrPDtp\nv+7AzsDWyQczs8QH9r8ktQcuBeoAV1cQp8sgT7yZkfjj2R/4BphS0yc0sxeB6UBZ7XsXSZoDzAdu\nBS4zs+XlHPLPcftJwA6EP9yyXG9ms8zsp3jsRFPL6cC1Zva1mS0DrgE6l1HrXRf4pawTSGoNdAX+\nZmaLzGw08ABwYtJm75vZy/F1PUpIdlWxFGgCtAMU4y8ttuOBh8zsUzNbDPwd2DWlTfo6M5sT35t3\ngM4VnPsx4Lj4AXosKR/aZjbTzJ41swVmNo+QIMtsr09ybfwZLUwtMLMvgf7A88BFwIkV/G64DPPE\nmxmPEtope/H7GktN6gv8E2hQStlNZtaM8NW1C3CjpIPLOdZTZtbMzNY3s25mNqqcbZNrqRMJNVII\ntbLb4gW6OYS2aAEblXKMmYRaY1laAbNiskk+V/KxpiY9XgA0qEqbppm9TahFDgCmSbpPUtMyYpqY\ntF8J4XWUF1PjCs79E/A94UNqXOo3AElrSbpX0kRJvwHDgWYVtNfD6j+j0jwCtAVeNrNxFWzrMswT\nbwaY2UTCV/lDgOdK2WQ+sFbS8w3LO1wlzvsG4Y/2zHK2sVjD+QD4Y7rHrkDrpMdtgJ/j40nA6TGB\nJ5aGZvZhKcd4E9gp0V5Zip+BdSQ1STlXVb9NlPszMLPbzWwHoAOhyeHiMmJaWXuX1IhQc6/uN5xB\nwIWU/qF9IbAVsLOZNSU0iUD4QIOyf18q+j26CxgKHCipa+XCddXliTdzTgG6mdn8UspGA0fF2svm\ncduyTAMq09/yn4SuU2WS1I7wtT1TV/kvjhd9WgN/Bf4b198D/D3pYtHaSur2lszM3gTeAJ6XtINC\nl7gm8eJU71jz+xC4VlIDSR0J79vjVYy5zJ+BpB0l7Ry/7s8HFgGlffV+AjhZUufYFn4N8D8zm1DF\nmBL+S2hnfqqUsiaEdt05ktYBrkgpr+zvC5JOJDQn9QLOBR6RVG7N3GWWJ94MMbMfzGxkGcX/JvRC\nmEb4ilde8ngQ2Dp+XR+cxnk/AD4upSjRM2I+8DrhAs29FR0vTS8Qun2NBl6KMWNmzwPXA/+JX4u/\nBMpr3jgaeJmQeObG7bsQasMQ2o7bEmqazwNXxFp+VZT3M2gK3A/MJjQlzARuSj2Amb1FuKj5LKF9\nejNCu2y1mNlCM3uztPZYQht6Q2AG4ULuqynltwFHxx4PFfZXltQmHrOnmZWY2RPASML747JEPhC6\nqwxJBmxhZt/nOhbn1lRe43XOuSzzxOucc1nmTQ3OOZdlXuN1ztUKks6WNFLSYkkDU8r+rHCr+zxJ\nX8W7MJPLz5c0Nd5S/lDs1ZIoayvpHUkLJH2j1W/vLz2W2lrjVXFDU70mFW9YC3Vu3ybXIeStWvrn\nkpaffprAzBkzMjqwUp2mm5gtK62zx+ps4fTXzOyg8raRdBSwgnAbd0Mz6xXXb0Toh38EodfIIcDT\nQFsz+1XSgYQ+1t1Y1cPmIzO7NO4/AhhB6Np5CKGXzxZmNr2sWGrtyEWq14T6W/0512HkpQ8+uiPX\nIeStRUtX5DqEvNWt684ZP6YtW5jW3+mi0QNaVLSNmT0HoDCYUvKNOxsDc8zslfj8pdgNczPgV8IY\nGg+a2di4/1WE7oiXStqSMGbJAbE74LOSzgN6EPq1l8qbGpxzeUygoooXaBGbERJLn0qcZCTwtaTD\n42ht3YHFwOexvAMwJmn7McAGktaNZT+m3No+Jq4vU62t8Trn1gACiioalgKAGWbWpSqnMLPlkgYR\n7kxsQLjR5k9Jd6E2Jtzgk5B43KSUskR5aeOTrOQ1XudcfpMqXqp1eO0H3ADsDdQjjP72gKTEyHIl\nhLsbExKP55VSliifRzk88Trn8ljaTQ3V0RkYbmYjzWyFmX0C/I8wZjSEMU6ShxztBEwzs5mx7A8p\ngzl1ooJxUTzxOufyW4ZqvHEgpgaEgd/rxMGXigkzfuyRqOFK2o4ws0eijXcQcIqkrSU1JwzHOhAg\nDrI/mjApQQNJRwIdCeN5lMnbeJ1z+Utkokab0JfVR3c7AbjSzPopTC76jKQNCBMMXGNmrwOY2auS\nbiAMbN+QkFSTj3MsIRHPBn4Cji6vKxl44nXO5TWle3GtQmbWD+hXRtmdxCmVyii/BbiljLIJhPbh\ntHnidc7ltxqf7Dr7PPE65/KYMtnUkDc88Trn8pfwGq9zzmWd13idcy6bBHUyc3Etn3jidc7lr8x2\nJ8sbnnidc/nN23idcy6bvFeDc85ln9d4nXMui5S5O9fyiSde51x+86YG55zLMm9qcM65bPKLa845\nl30FWOMtvI8S51zhkKCouOIlrUPp7DgR5mJJA1PK1pJ0l6QZkuZKGp5UJknXS5oZlxukVZ8GkjpL\nGiVpQfy/MxXwxOucy2+Zm3PtZ6A/8FApZfcB6wDt4//nJ5X1AboTpvTpCBwKnB5CUz3gBeAxoDnw\nCPBCXF8mT7zOufyWoTnXzOw5MxsMzFzt8NJWwOFAHzObbmbLzWxU0iYnATeb2WQzmwLcDPSKZXsT\nmmxvNbPFZnY74UbnbuXF4onXOZff0qvxtojNCImlTyXOsDMwEbgyNjV8IalHUnkHYEzS8zFxXaLs\nczOzpPLPk8pL5RfXnHP5S2n3aphhZl2qeJaNgW0Ic6m1AnYFXpL0lZl9DTQG5iZtPxdoHNt5U8sS\n5U0oh9d4nXN5TUVFFS7VtBBYCvQ3syVm9i5hYssDYnkJ0DRp+6ZASazlppYlyueVd0JPvDXkniuO\nZ+Jb1zLy6X+sXNdxy41495EL+eg/l/L+45fQpcMmAGzZdgOGPXIhc/73b847cd/VjrN244Y8ceMp\njH6uL58925edO26a1deRC8uXL2eXHbfnqO6HATBh/Hj23H0Xtt16S078y7EsWbIkxxFmx9lnnMqW\nm7Rkty6dVq6bPWsWRx56IF06tuPIQw9kzuzZK8veHz6MPXfZgV27dOTQA/fJRcgZFyagUIVLNX1e\nQflYwoW1hE5xXaKso1YPomNSeak88daQR4d8xBFnDVht3dXndefq+15hl2Ov46q7h3L1ed0BmD13\nPhde/zS3Dnr7d8e56ZKjef3Dr+h8VH92OuZavvlxalbiz6UBd9xGu3btVz7v+49LOefc8/jiq+9o\n1rwZAx9+MIfRZc9fTujJ04NfWm3drTdfz157d2Pk59+w197duPXm6wGYO2cOF51/Dk88/TwjRn7O\nw4/+NxchZ57SXNI5lFQsqQFQB6gjqYGkYmA4YVr2v8dtdidcNHst7joIuEDSRpJaARcSpnMHGAYs\nB86VVF/S2XH97/+Yk3jirSEffPoDs+YuWG2dGTRt1AAINdlfpoemoemzSxj11U8sXbZ8te2bNGpA\n1+03Y+DzIwBYumw5c0sWZiH63Jk8eTKvvvIyvXqfAoCZ8e6wtzmyx9EAnHDiSQx98YVchpg1u3Xd\nk+brrLPauldeGsKxx/cE4Njje/Ly0BcBeOapJzns8O5s3LoNAOutv352g60xFdd2K1Hj7UtoVrgU\nOCE+7mtmS4EjgEMI7bP3Az3N7Ju4373AEOAL4EvgpbgOM1tC6GrWE5gD9Aa6x/Vl8otrWXTxTc8w\nZMBZXHv+kRQViX163Vzu9ptutC4zZpdw35UnsO2WG/HZ15O46IZnWLCocL9qX3Lh+fS/9npK5oUm\nspkzZ7J2s2YUF4df1Y022pifp0zJZYg59euv09iwZUsANmzZkunTfwXg+3HjWLZsKYcd1I2SeSWc\nfuY5HHv8ibkMNWMy0JQAgJn1A/qVUTaWcFGttDIDLolLaeWfATtUJpa8qPFKmiBpoaSSpOXOpPK9\nJZmkS1L2axvXJ/aZIOnS7L+C9PT50x5ccvNzbHHwZVxy07PcfcXx5W5fXFyHzu1ac//T77Hrcdez\nYOFiLuq9f5aizb6XXxrKeuuvx/bbr/odXr2XTpCpP8RCsnz5MkZ/Nor/PDuEZ154mZuuv5rvx32X\n67AyoqioqMJlTZNPER9mZo2TlrOTyk4CZsX/S9PMzBoDxwGXSzqopoOtiuMP3ZnBb40G4Nk3Plt5\nca0sU6bNZsqvc/jky4kAPP/maDq3a13jcebKRx9+wEtDh9Bui03pecJxvPvO21xy4fnMnTOHZcuW\nATBlymRatmqV40hzZ/31N2DqL78AMPWXX1hvvdCk0KrVRuy7/4E0atSIdVu0YNfd9+DLLyq6ZrQG\nyGAbbz7Jp8RbKklrAUcDZwFbSCqzr56ZjSBcTdwmS+FVyi/T57LHDlsAsPdOW/L9T9PL3X7azHlM\nnjqbLTZZP+6zVUFfXPvX1dfy/fhJfDNuPIMee5K99unGw4MeY8+99uH5Z58B4LFHH+GPhx2e40hz\n56BDDuU/jw8C4D+PD+LgP4aeHwcfejgfffA+y5YtY8GCBYz65GO23KpdLkPNCGW2jTdvrAltvD0I\nfeWeBo4nNGKPTN0odufYjXDHyGelHSjezRLuaKnbuGaijR65thd77LAFLZo15vtXr+Kqe17mrKue\n4MaLj6a4uIjFi5dxdv8nAdhg3SZ88PglNGnUgBVmnH383mzX42rmzV/EBdc/zcPX9KJecR0mTJlB\nnyseq9G481H/a66j5wnHcWW/y+jUaTt6nXxKrkPKilNPOp4P3nuXmTNn0GGLTbi07xWcd+Hf6H3i\nsTw26GE23rg1Dz8Wei9s1a493fY/kK47b0eRijixV2+27pCX9Y9KWxMTa0VUWhta1oOQJgAtgGVJ\nqy82s/slvQl8aWbnSToOuB1oZWZLJbUFxhOuRBowFbg73i9drqK11rf6W/05sy+kQMz6+I5ch5C3\nFi1dkesQ8la3rjvz2acjM5oli9f9gzU9pH+F281+7PhR1bhzLevyqcbb3czeTF4hqTWwD/D3uOoF\nwihCfwQGJ23awsySk7ZzrhAIVFR4Nd58SrylOZHQDj0k6etGA0Jzw+CydnLOFY5CbGrI98TbE7gS\nuCdp3U7A05LWzU1IzrlsSVxcKzT5lHiHSEq+desToC0wwMySL/+/KOl7QtexoVmMzzmXA554a4iZ\nta3k9sljXRbeT8U5t0oB/oXnReJ1zrlSiTXyzrSKeOJ1zuU1b2pwzrks8otrzjmXC4WXd/N/rAbn\nXC2mzM1AIensOBHmYkkDy9jmijji4X5J6+pLekjSb5KmSrogZZ99JX0jaYGkdySVP/oVnnidc3ku\ng8NC/gz0Bx4qrVDSZoQBuX5JKeoHbAFsQriT9pLECIiSWgDPAZcB6xDGkalw+g9PvM65/JahYSHN\n7DkzGwzMLGOTO4G/AakzDfQErjKz2XHW4fuBXrHsKGCsmT1tZosISbqTpHKHhvPE65zLa2k2NbSI\nzQiJpU8lz/EnYImZvZyyvjlhyvcxSavHEEZBJP6/sszM5gM/JJWXyi+uOefyViXacGdUdXQySY2B\na1g1nXuyxPixc5PWzQWaJJWnDqydXF4qT7zOubyWhe5kVwKPmtn4UspK4v9NgUVJj+cllTdN2Se5\nvFTe1OCcy2sqUoVLNe1LmJ59qqSpQGvgKUl/M7PZhIttnZK270SY6Yb4/8oySY2AzZLKS+U1Xudc\nXstUjVdSMSHn1QHqSGpAmHxhX6Bu0qafABcAr8Tng4C+kkYCGwCnASfHsueBGyX1IEz7fjnwedLU\n8KXyGq9zLn9lsB8v0BdYCFwKnBAf9zWzmWY2NbEAy4HZZpZoZriCcMFsIvAucKOZvQoQR07sAVwN\nzAZ2Bo6tKBCv8Trn8paATDXxmlk/QnevirZrm/J8MdA7LqVt/yZQqZlFPfE65/KYj9XgnHNZV+Rz\nrjnnXBYpc00N+cQTr3Mubwmv8TrnXNZ5jdc557LML64551wWSd7U4JxzWebdyZxzLusKMO964nXO\n5Tev8TrnXDZ5P17nnMsu78frnHM54E0NzjmXZQWYd2tv4m2/+cY8+eK1uQ4jL108tNwxnGu1/gdt\nmesQahdldCD0swmzA28LPGlmveL6XYCrgB0IY/EOA841s19iuYDrgFPjoR4E/mZmFss7x3Xtga+B\nU8xsdHmxlDkQuqS10l2q9C4451wFEuPxVrSk6WegP/BQyvrmwH1AW2ATwnxpDyeV9wG6E6b46Qgc\nCpwOIKke8ALwWDzOI8ALcX2ZyqvxlgCWzqshTKXhnHMZpoxdXDOz5wAkdQE2Tlr/SvJ2ku4kzDSR\ncBJws5lNjuU3E6b/uQfYm5BHb4014NslXQR0A14tK5byEu/B6b8k55yrGWk2NbSIc6Il3Gdm91Xx\nlHuy+mSVHYAxSc/HxHWJss8TzQ7R53F95ROvmb1W2Widcy6j0m9KmGFmXap9OqkjYcLKI5JWNwbm\nJj2fCzSObb+pZYnyJuWdJ+3JLiWtK+lsSf+WtG5ct6Ok1ukewznnKiO08WZsssvyzyVtTphZ+K9m\n9l5SUQnQNOl5U6Ak1nJTyxLl88o7V1qJV1In4FvgrLisHYsOB65J5xjOOVcV2Ui8kjYB3gSuMrNH\nU4rHEi6sJXRiVVPEWKCjVg+iI6s3VfxOujXem4EHzKw9sDhp/SvAHmkewznnKq2oSBUu6ZBULKkB\noTNAHUkN4rqNgLeBAWZ2Tym7DgIukLSRpFbAhcDAWDaM0AXtXEn1Y5c14vHKlG4/3h2J3SdSTAE2\nTPMYzjlXOZkdq6EvcEXS8xOAKwm9t/4AXCFpZbmZNY4P743lX8TnD8R1mNkSSd3juusI/Xi7m9mS\n8gJJN/Eu5vftGABbAjPSPIZzzlWKMjger5n1A/qVUXxlOfsZcElcSiv/jHDzRdrSbWoYAvSVlEjU\nFqvn1wHPV+aEzjlXGRm8gSJvpJt4LyJ0OJ4KNCS0X/wILAH+UTOhOeccFEkVLmuatJoazGy2pF2B\ng4DtCQn7U+BlM1tRg/E552qxWj/nWkywL8fFOeeyogDzbqVuoDhI0uuSJsflDUl+W7FzrkZl6waK\nbEr3BopzgBeBaYQbJq4htPcOlnRuzYXnnKvtCvHiWrpNDZcC55vZgKR1d0kaQegbd3vGI3PO1Xoi\ndCkrNOk2NawNlDZozuuU3r/XOeeqT6JOUcXLmqYy/XiPKGX9EfjFNudcDapVTQ2Szkx6+gXhBoo9\ngI/iul2AvYAbai4851xtJlgj++lWpLw23stSni8Cdo5L8rpzAJ+8zDlXIwow75Y7EHrLbAbinHOl\nWRO7i1Wk1s4y7JzLfxJr5MWziqSdeCW1BY4C2gCrzaBpZmeWsotzzlVb4aXd9G+g2J8wzuSJwJmE\n8XmPAf4CbF5j0Tnnar1M3bkWpy4bKWmxpIEpZftK+kbSAknvxBkpEmX1JT0k6TdJUyVdkO6+ZUm3\nO9k1wHVmth1hbN5jCDXfdwl3tDnnXMaFXg0VL2n6GegPPLTaOaQWwHOEDgXrACOB/yZt0g/YAtgE\n2Ae4RNJBae5bqnQTbzvgsfh4GdDQzOYTZuO8OM1jOOdc5aRR2023xmtmz5nZYGBmStFRwFgze9rM\nFhESbSdJ7WJ5T8JcbLPN7GvgfqBXmvuWKt3EO59V7bq/AJslXguwbprHcM65SktzzrUWsRkhsfSp\nxCk6AGMST2Kl8gegg6TmQKvk8vi4Q0X7lnfCdC+ufQzsRmjnfRW4UVJ7oEcsc865jEs0NaRhhpl1\nqeJpGgPTU9bNBZrEssTz1LKK9i1Tuon34qQArgCaA6cA3xFuoHDlWLxoESf/6SCWLlnCsmXL2P+Q\nIzjzwn/Sq8eBLJhfAsCsGdPZpvMO3PrAk3wy4j3OO/U4Nmod2ui7HXQYZ5x3aS5fQo3ae7Pm7Na2\nGUJ8MGE2w36YzXatmnBI+/XYoEk9bho2gZ/mLAKgjuC47VrSplkDVgDPfj6NcTMW5PYFZMk9A27n\nkYcfBDN6nnwK/3f2X7n6yst5+aUhFKmI9dZfjwH3PkTLVq1yHWpGZaEfbwm/H3OmKTAvliWeL0op\nq2jfMqU7A8W3SY/nAScDSKoLtE7nGLVZvfr1eeA/Q1mrUWOWLl1Krx4H0HWf/Rn47Kpxhy44/QT2\n2f+Qlc+323FX7hz4dC7CzaqWTeqzW9tm3DhsAstXGGfu1oaxU0v4ed5i7v/fZI7rvPok1ru3bQ7A\nNW+Pp3G9Opy5W2tuHDYBy0XwWfTV2C955OEHeWv4COrVq8fRRxzCAQcdwjnnX8Q/r/gXAPfedQc3\nXNuff99xV46jzawsdCcbC5y08nxSI0Jz6tg4+84vQCfgjbhJp7hPufuWd8K0B0Ivw9bAuGoeo+BJ\nYq1G4QvDsmVLWbZs2Wr3Qc4vmcfHHwxnnwMPzVWIObNhk3pMmLWIpcuNFQbfz1hAp1ZNmDZvCb+W\n/H6G7A2b1uPb6fMBKFmynIVLV9CmeYNsh5113337DTvuuDNrrbUWxcXF7N51T4a+OJimTVdVtubP\nn19wd3lJmZtzTVKxpAZAHaCOpAZxAt/ngW0k9YjllwOfm9k3cddBhLFqmseLZqcBA2NZRfuWqrqJ\n16Vp+fLl/Pmg3dlnu83Ypes+dNxux5Vlb786hJ1334vGTVb9EX3+6cf86cDdOLPnUXz/7de5CDkr\nfp63mM1bNKRRvTrUrSM6bNiI5g3rlrn9lLmL2bZlE4oE665Vl9bNGpS7faFov3UHPvzgPWbNnMmC\nBQt447VXmDJ5MgBXXdGXDlu05en/Psk/LuuX20BrQJoX19LRF1hIGF/8hPi4r5lNJ1yvuhqYTRiP\n5tik/a4gXDCbSOhCe6OZvQqQxr6lv6Z0I65JkiZI2i9lXS9J7yeVL5RUImmapIclNY5lwySdmou4\nK6NOnTo89eoHvP6/r/lyzCjGffvVyrJXXnyGg484euXz9tt04tURY3n6tQ85rtfpnH/acbkIOSum\nzVvCG9/N5Ozd23DWbm2YMncxy63shoMRE+cwZ+FSLtl7U3p03IDxsxaWu32h2Kpde/56wcUceehB\nHH3EIXTYthPFxXUAuOzK/owdN4E/HXMc998zoIIjrXkyNSykmfUzM6Us/WLZm2bWzswamtneZjYh\nab/FZtbbzJqa2QZmdkvKccvctyx5kXjTdJiZNSbMcrwj4dNrjdN07WbsuEtXPhz2JgBzZs/ky9Gj\n2KPbgSu3adyk6cqmiT26HciyZcuYPSu162HhGDFxLte/M55b35vI/CXLmV6ytMxtVxg898WvXPfO\neO77aDIN6xYxvZQmiUJ0Yq/evDviE15+YxjNmzfnD5ttsVr50cccx4svPJ+j6GqGqLiZYU0cNrLc\nxCtp6/IWVvXnzRozmwK8AmyT7XNX1ayZM/ht7hwAFi1ayEfvD6Nt/KN5fehg9tz3IOo3WNVOOePX\naVisxX0xeiQrVqygWfN1sh94ljSuF2puzRsW06lVE0ZOnlvmtnXriHp1wh9au/UascJg6rzakXin\n//orAJMm/cTQFwdz9J+P5YfvV11iefWlIWy55Va5Cq9mpFHbXQPzboW9Gr6Eci8Yq4LyjJPUGjiE\ncJteZfftA/QBaLlR9jpjzPh1Kn0vOIMVy5ezYsUKDjj0SPbaL0zQ/NqQZ+l95vmrbf/Gy4N56tEH\nKS4upn6DBlx/58MFd9Ek2ak7b0yjenVYbsZTY6aycOkKOrZswp86bUDjenU4Y9fWTJm7iAEfTqJJ\n/WLO2q01BsxZuIxHRk7JdfhZ0/Mvf2L2rFkU163Ljf++nWbNm3PumX0YN+47ioqKaN26DbfcXlg9\nGqAwh4WUldM+Jimtj8/k7mZVCkKaALQg3I6cUA/41My6ppTPBV4CLjSzhZKGAY+Z2QOVOWeHjtvb\nky+9W52wC9Z9IyfnOoS81f+qOkivAAAeMElEQVSgLXMdQt7aZ/ed+ezTkRnNkhtsvo0dc9MzFW53\nx5HtR1XjBoqsK7fGW92EWkndzezNxBNJvYBTyyp3ztUOBTgcrw+E7pzLb55481eiY3TCcjMr+9K4\nc26NEC6eFV7mXZO6k5XnbkJn6MTycG7Dcc5lSgbH480beVHjNbO2pawbSLwtr7TypO32rpmonHO5\nJgpzzrVK1XglNZbUKQ6O45xzNa4ojWVNk+6ca40kDQJ+A0YRRySTdKekf9ZgfM65Wq4Qb6BI98Pi\nWsL0P7uxakxKgNeBP2U6KOecg3BhrRBvGU63jfcI4M9m9j9JyXdcfAX8IfNhOedcsAbm1QqlW+Nd\nD/i1lPWNMhiLc86tRkBxkSpc0jqW1FbSy5Jmx2na74zj8SKps6RRcYr2UZI6J+0nSddLmhmXG1TN\nPm7pJt5RhPEREhK13t7AiOoE4Jxz5clgG+9dhApkS6AzsBdwpqR6wAuEmdSbA48AL8T1EMZ36U6Y\neaIjcChwenVeU7qJ95/A9ZLuIDRPnCXplRjQZdUJwDnnypRGH95K9DbbFHjKzBaZ2VTCxL0dgL0J\nee3WOPbu7eHMdIv7nQTcbGaT4+iIN7NqevcqSSvxmtlwwqfD+sAUwlzy84HdzcxnGXbO1Ril8S9N\ntwHHSlpL0kbAwaxKvp/b6iOGfU4ZU7iz+vTuVZL2DRRmNgo4pjonc865yqjE9O4tJI1Men6fmd2X\nss27hPnSfiPMu/YIMJgwqULqINCpU7inTu/eWJKsvOEdy5FW4pW0VnnlZlY75td2zmVdmneuzShv\nWEhJRcBrwL2EbrGNgYeA64FfKH+K9tQp3JsCJVVNupB+G29JDKKsxTnnMi5R481AG+86hBu/7ozt\nuDMJY7ocQpiKvWNKT4WOrD6Fe6eksuTp3ask3aaGg1Oe1wW2I4yX6xfXnHM1I0N3ppnZDEnjgf+T\ndBOhxnsSob12GLAcOFfSPYTmCIC34/+DgAskvUzo0XUhcEd14kkr8ZrZa6WsHirpO8I0yYOqE4Rz\nzpUlg3emHQXcCvyNkGjfAc43syWSugMPANcBXxMmXkhM5ncv4UaxL+LzB+K6Kqvu6GQjCe0kzjmX\ncZW4uFYhMxtN6DpWWtlnwA5llBlwSVwyosqJN3YuPovQvcw552qAqFOA9wyn26thOqvPJiygGbAE\n6FkDcTnnHKIwx2pIt8bbN+X5CmA68KGZlTaGg3POVd8aOsNERSpMvHEQiaXAy/E2O+ecy5o1cdjH\nilTYj9fMlgF3AvVrPhznnFsl0dRQaAOhp9vU8DGh0/DEGozFOed+pxDnXEs38d4J3CypFWGIyPnJ\nhWb2VaYDc845sWbOqVaRdBPvU/H/u+L/iR4Oio/rZDIo55wD4p1rtbfG275Go3DOuTIUXtqtIPFK\negj4q5l9m6V4nHNupXDnWuGl3oqaT04CGmYjEOecK00GZ6DIGxU1NayBL8k5VzhUa9t4qzzYr3PO\nVUdt7tUwtaJPHDPzXg3OuRpRW2u8fYA5NR2Ic86VJpNpV9KxwBVAG2Aq0MvM3pO0LzAgrv9fXD8x\n7lMfuBs4GlgA3GBmt1QnjnQS75BCHAhHgrp1CvFLTPXddJj3HizLfre+l+sQ8ta46SUZP6ZExoaF\nlLQ/YY61Ywh347aM61sAzxFm1BkCXAX8F9gl7toP2ALYBNgQeEfSV2b2alVjqSjxevuucy6nMtjU\ncCXwLzP7KD6fEo/fBxhrZk/H5/2AGZLamdk3hKFvTzaz2cBsSfcDvQhTw1dJRVW+wmtccc6tUZTG\nUuExpDpAF2A9Sd9LmizpTkkNgQ6EudcAMLP5wA9AB0nNgVbJ5fFxh+q8pnJrvGbm38WdczmVZoW3\nhaSRSc/vM7P7kp5vQJik92hgD8JQty8QxhpvTBhfPNlcoEksSzxPLauy6s655pxzNSZ0J0sr884w\nsy7llC+M/99hZr8ASLqFkHiHA01Ttm8KzANKkp4vSimrMq/ROufymChSxUtFYvvsZEq/bjWWMOxt\nOKPUCNiM0O47G/gluTw+HluNF+WJ1zmX3zI4EPrDwDmS1o9tt+cBQ4HngW0k9ZDUALgc+DxeWAMY\nBPSV1FxSO+A0YGB1XpMnXudc3ko0NVS0pOkq4BPgO+Br4DPgajObDvQArgZmAzsDxybtdwXhYttE\n4F3gxup0JQNv43XO5bMMTu1jZkuBM+OSWvYm0K6M/RYDveOSEZ54nXN5rQDvGPbE65zLXyJzd67l\nE0+8zrm8pgK8j8sTr3MurxVghdcTr3Muv3mN1znnsijMuZbrKDLPE69zLn+leWfamsYTr3MurxVe\n2vXE65zLY4U6vbsnXudcXiu8tOuJ1zmX7wow83ridc7lNW9qcM65LCu8tOuJ1zmX7wow8/p4vM65\nvBUms6z4X6WOKW0haZGkx5LW/UXSREnzJQ2WtE5S2TqSno9lEyX9pbqvyxOvcy5/pTH7RBWagAcQ\nBkQPp5A6APcCJxImxVwA3JWy/ZJYdjxwd9ynyrypwTmX1zJ5bU3SscAc4ENg87j6eGCImQ2P21wG\nfC2pCbCCMDvFNmZWArwv6UVCkr60qnF4jdc5l8fSaWgQxOndk5Y+vzuS1BT4F3BhSlEHYEziiZn9\nQKjhbhmX5Wb2XdL2Y+I+VeY1XudcXkuzxlvR9O4Q5lx70MwmafWDNgbmpmw7F2gCLC+nrMo88WbB\n4kWL6NnjQJYsXsyy5cs44I/dOeeivlx8dm/GjvmM4rrFbNu5C/2uv526devy1mtDuePGq5CKKC4u\n5tIrr2eHnXbL9cuocZMmTeLUk3sybdpUioqK6H1KH84+9698PmYM55x1BvNLStikbVseHvQ4TZs2\nzXW4NeLvB27Bbputw+wFS+k58FMA9tmyBb13a8Mm667FaY+N5ttpJQAUF4mLD9icdhs0wcy47Z0f\n+WzS6jniuu5b06pZg5XHWtOIzHRqkNQZ2A/YrpTiEiD1F6opMI/Q1FBWWZV5U0MW1Ktfn4eeeonn\n3/yI514fwfvD3mTMqI859MhjeGn4p7zw1scsXrSQZ58YCMAuXffm+Tc+4vk3RtD/5ru5/KKzcvsC\nsqS4uJjrbriZ0V98zbvvf8S99wzg66++4v9OP5X+11zHyNFfcPgRR/Lvm2/Mdag15uWx07jwmS9X\nW/fjjPn844WvGTN59aR6eMcNATjpkU8575kvOXuvTVdLUntusS4Lly6v6ZBrntJYKrY30Bb4SdJU\n4CKgh6RPgbFAp5Wnk/4A1CfMRvwdUCxpi6RjdYr7VJkn3iyQRKNGjQFYtmwpy5YuBYm99j0QSUhi\n285dmPrLFAAaNWpM4qvQwgXzUQHeuVOali1bst322wPQpEkT2rVrz88/T2Hcd9/SdY89Aei23/4M\nfv7ZXIZZo8ZM/o3fFi1bbd3EWQuZNHvh77Ztu+5ajJo4B4A5C5Yyb/Fy2m0Yfs8a1i3i2B024pGP\nJtV80DWsKA4NWd6ShvuAzYDOcbkHeAk4EHgcOEzSHpIaEdqBnzOzeWY2H3gO+JekRpJ2B44AHq3W\na6rOzi59y5cv58j9d6Vrx03Zbc9udNp+x5VlS5cu5cVnn6TrPvuvXPfmKy/yxz2344yTjqb/zXfn\nIuScmjhhAqNHf8aOO+3M1h22YeiQFwF47pmnmTxpzU8mmfD99Pnssfm61BG0XLs+W23QmPWb1Afg\n1N3b8p+RU1hUADXeTFR4zWyBmU1NLITmhUVmNt3MxgJnEBLwr4T22+Qp4M8EGsayJ4H/i/tUWdYS\nr6QJkpZIapGyfrQkk7SrpN8k1Ukqu7+MdffExx0kvS5ptqQ5kkZJOiRbr6ky6tSpw/NvjOCdkd/y\nxWcjGffNqp/bVf84ny47706XnXdfuW6/gw/npeGfceeDT3L7jVflIuScKSkp4bg/9+DGm2+ladOm\n3Hv/Q9x79wB222kHSkrmUa9evVyHmBde+mIqv85bwgMnbse5+2zGlz//xvIVxubrNWLj5g0Y/v3M\nXIdYfelk3Sp8ITSzfmZ2QtLzJ8ysjZk1MrMjzGxWUtksM+sey9qY2RPVek1k/+LaeOA44A4ASdsS\nPkkAZgB1gO1Z1bl5D+DnlHV7AlfGx0OAu4FD4/MdyfMbDJuu3Ywdd9uD94a9yRbtOjDglmuYNXMG\ntz9Q+s+yyy5dmTRxPLNnzaD5Oi1K3aaQLF26lOP+3INjjjue7kceBcBW7dox9JXXARj33Xe88vJL\nuQwxbyw3uGPYjyuf331cJybPWUTnjddmqw0a8/RpO1KnSDRfqy53HLMt5/z3ixxGW3WFOOdatpsa\nHgV6Jj0/CRgUHy8FPiIkViStD9QD/puybktgeKw5bwrcb2ZL4vKBmb2flVdSCbNmTue3uaEtbtHC\nhYx47x3+sNmWPPPEQD4Y9hY3DXiYoqJVP4qJ43/AzAD46ovRLF26hGbN181J7NlkZpxx2ils1a49\nfz3/gpXrf/31VwBWrFjBddf057Q+Z+QqxLxSv7iIBnXD702XTZqxfIUxYeYCBo/5he73fMyf7v+E\nM58cw6TZC9fgpFsjd67lXLZrvB8BJ0pqT7haeAzQFegfy4cTkuzN8f/343J20rrxZjZZ4YrT98Bj\nkh4ARpjZtPJOHjtV9wFouVHrDL+0sk2fNo2/n9eHFSuWs2LFCg467Cj23v9gtm2zNq02bsNxh3cD\nYP9DDufM8//OGy+/wAvPPEFxcV0aNGjIzXc/UisusH34wQc88fijbLPNtuy8Q2cArux/Dd+PG8e9\n9wwA4IjuR9Gz18m5DLNG9fvjVnRu3YxmDYt57vSdePCDicxbtIzz9t2MZg3rcuNRHRj363wufPZL\nmq9Vl1uO3oYVBjNKFnPVK9/mOvwaUYi/+krUrGr8RNIE4FRgF6AR8C7hDpKDCbXdTQndPZ4FWgC3\nAl8SGrN/ADaM65qa2cnxmBsTbts7OO7/PnCKmY2rKJ5tOm1vT7/yXsZeXyHZdP1GuQ4hb+13q//O\nlGX0badRMumbjKbJbTptb8+8WvGX2PatGo1K4waKvJGLXg2PAn8BerGqmSHhI8JdJNsQarfvxfuj\nJyWtG57Y2Mwmm9nZZrYZsAkwv5RjOufWYIXY1JD1xGtmEwkX2Q4h9I9LLltEuIh2KNDSzL6JRe/F\ndR1JSrwp+04ijCK0Tc1E7pzLhRro1JBzuerHewrQLXZOTjUcOI8welDC+3Hd1DiABZKaS7pS0uaS\niuLFtt6EWrNzrlAUYObNSeI1sx/MbGQZxe8C6xOSbcL7cV1ybXcJoU34TeA3QnvwYkIThnOuAEgZ\nu3Mtr2StV4OZtS1j/TKSPrPM7DVSPsNib4XUdfMJ3dGccwVszUurFfPRyZxz+a0AM68nXudcHqv8\nnGprAk+8zrm8tgY24VbIE69zLm8lbhkuNJ54nXN5rRCbGnw8XudcXsvEnWuS6kt6UNJESfMkfSbp\n4KTyfSV9I2mBpHckbZKy70NxiNqpki4o/Szp88TrnMtrGbp/opgw9MBewNrAZcBTktrGm6+ei+vW\nAUYSRkVM6AdsQRiWYB/gEkkHVec1eVODcy5/iYyMzBf7/fdLWjVU0nhgB2BdYKyZPQ0gqR8wQ1K7\nOGxBT+BkM5sNzJZ0P+FGrVerGo/XeJ1zeasS4/G2kDQyaelT7nGlDQhje48FOgBjEmUxSf8AdJDU\nHGiVXB4fd6jO6/Iar3Mur6VZ352R7rCQkuoS5ld7xMy+kdQYmJ6y2VzC3GuNk56nllWZJ17nXF7L\nZHcySUWEoWmXECZYgDDxZdOUTZsC82JZ4vmilLIq86YG51xeUxr/0jpOaCx+ENgA6GFmS2PRWKBT\n0naNCFPBj43tur8kl8fHa8Ysw845VxUZHAj9bqA9cJiZLUxa/zywjaQekhoAlwOfJ40HPgjoG4ei\nbQecBgyszmvyxOucy1vpJN00+/FuApwOdAamSiqJy/FmNh3oAVwNzAZ2Bo5N2v0KwsW2iYRha280\nsyr3aABv43XO5blM3LkWZ74p80Bm9ibQroyyxYRJFnpXO5DIE69zLr8V3h3Dnnidc/mtAPOuJ17n\nXD5bM6f2qYgnXudc3irUYSG9V4NzzmWZ13idc3mtEGu8nnidc3mtEAdC98TrnMtbEhQVXt71xOuc\ny3OeeJ1zLru8qcE557LML64551yWFWDe9cTrnMtvmZhzLd944nXO5a1CvXNNZpbrGHJC0nTC+Jr5\noAUwI9dB5Cl/b8qXT+/PJma2XiYPKOlVwmusyAwzq9aU69lUaxNvPpE0Mt2J+mobf2/K5+/PmsnH\nanDOuSzzxOucc1nmiTc/3JfrAPKYvzfl8/dnDeRtvM45l2Ve43XOuSzzxOucc1nmidc557LME28e\nUiHeI+mcW8kTbx4yv+LpqkjSYZLa5ToOVz4fqyGPSDoN2BD4HzDSzGblOCS3BpH0X2A94GpJ481s\nca5jcqXz7mR5QtJgYCOgBFgfuM3M7pMkrwG7isSk2wr4I7DQzJamlBeZ2YqcBOd+x5sa8oCkIUBD\nM9vRzPYB3gVOl1Rc25OupKMldch1HPlMUhegDXCQmf1mZkslrSdpL0n7A5jZCkn+954nvKkhxyTt\nQ6il7J20+gVC7bcRMDcHYeUFSdsAVwEvSnrQzL7LdUx5yoAmZjZf0vrA9sAgYDbQXNJwMzvaa7z5\nwz8Bc8zM3gGuAF6T1FlSXeB+4G0zq7VJF8DMvgR6ArsAp0jaMlEmqU7S47VzEF7OSTpfUm8zGwU0\nkPQ58DDwb+AOYC/gL8BOiZqvyw9e482hRLubmV0VvwaOBBYC/zCzO+I2ta6NV9I1wGbAdOBW4K+E\nRGKSHjazb81sedz2r0AnSWeY2ZKcBZ1lkhoBOxMqTw8Rarn9gTHAeDN7O263gDDu9LwchepK4RfX\nckDSDsB3ZjYvZf05wG3ANmb2laS6qRdJCp2kFwgDX78FdCYklu6E2tsVwAjgITP7VtJFwA3ADmb2\nWY5CzhlJmwGjgBvN7OoytjkdOA/Yz8ymZDM+Vzav8WaZpP7AP4D3JV0NfG5mvwCY2R2S1gNGSzrQ\nzN6pTVejJT0DbJQY2FtSd+A6oKWZvSXJgH7AnNiWeQawY21KupLOAB4AiszsB0n/B/STNCJRy43b\ndQV2By4GDvCkm1888Wbfl8B4YBYwEPhM0tvAncBiM7tc0nLgLUl7mtn7uQs1eyStAxwI3CZpLTNb\nAHwFLAAaxyaXt+NNfbcTLj52je2btYKko4C7CG3eb0oaDjxNeN8OljTKzObGZoiD43b7mNkXOQva\nlcqbGrJEUgMzWyTpD8A7wEXAp8AJQC9Ce+YrhK+NJZLOBt4ys69zFXO2SLqc0IQwGXiR8IH0CjAY\nuNPMbkjZvgthjq0J2Y00tyS1Jrwn8wkJ92BCjXZjwgd3LzP7IG7bkNBF0W/CyUPeqyELJN0PHA9g\nZj8CNxHaJleY2ZWEJLwDcDTwi6QLgLtrSdJtDjQErgbWAY4AegMfAo8nkm5yH1QzG1nbki6AmU0C\nzgHqAD8CTwFDgHrAz8AASU3itgs96eYvT7w1LF4s2s7MHkxa/Q4wDWgjaU9C958zgW0J7b+DE1ft\nC52ZzQYGEPou3wI0Bw4AphDacteJ29WKdu5Uki6TNELS5rEJ5kNgKLC1mQ0Ezgb2I8w03JHwe+Ty\nnDc11CBJzwPrmVnXpHX1zGyJpLuAEwm3CP/TzB7KVZy5EO/KW5b0vC5wDXAocDLwG6E29yTh9unp\nOQk0hyT9jXAX49VAXeBN4G7CeB7XAeeb2XeS2gBbELqTnWxm3+QoZJcmT7w1RNIDhIsebZP6nP6B\ncGHobGAx8AYw3MzOrE39dSVdS7jI+AywxMxMUitCm/eXQBPgQkItbgShJnx1bar1xjb+24HXgT7A\nYcC+QHvgWMKNJbsBeyX6L9emHjBrOm9qqDmPEq7IHwAgaWNgODA6tk/OAT4j1F5q21CQ7wCXA4fF\npNuaMCLbbYQPq8GENvCWhJsEnqyFCeUl4D1gJ0LTwkuE7nMvEZLxLEKzzBmJ8Ztr4Xu0xvLEW0PM\n7F1CTeWW2PdyBHCXmfWNtduFhGSzj6RmtWnwczN7HTgNuCq+Nx8C95jZtfHbwRPA20BfYJKZfZ+7\naLMv3g79MyHJnkT4gB4ONDKziwjfmNoDWwJHAQ1yFKqrIm9qqGHx4tnLwFAzOzZp/fmEr4vHmtm3\nuYovl0p7bxJfl2PTw2Izm5nTILNI0q7Ax0lNUycD5xNqvfcSLqJ1NbPxkjYi9NP9srb+/qzJPPFm\ngaTdCXcb/dPMnou1vGsJt3HWmhsAShPfm/uBfwKvxRsnah1JAwkfxA8QerxcET+ArgN+MbPbFMZs\n7ki4E61WfQsoNJ54s0TSXoSBXkYThoHc38w+zW1U+SG+N3cRrso/W5sGu0mQtBuh7XYQsCkg4DVC\nO3c9MzsvbvcG4brAdsm9QtyaxRNvFsWxd58gDFg9Jtfx5BNJ+xG+BXRLHTyotoi1/3uACwgDmzch\ntHOvAxxoZm/E7TY2s8k5C9RVmyfeLEvcOpzrOPJR0hgNtVb8cL4duNzMnpfUGdiccAv1Qu+5UBg8\n8TqXZyTtTWj3/ruZPZPjcFwN8NHJnMszZjZM0qnA3fGOvlrZ7l3IvMbrXJ7ydu/C5YnXuTzm7d6F\nyROvc85lmd8y7JxzWeaJ1znnsswTr3POZZknXuecyzJPvM45l2WeeF2VSfpSUr+k5xMkXZSDOLpI\nMklta/Ac/SR9mYHj5OQ9cvnFE28BkTQwJiCTtFTSj5JuktQoSyHsSBhlrEKSekkqqeF4ks83TNKd\n2Tqfc+XxW4YLz5uESTTrAnsQxndtBPxfaRtLqmtmSzNx4to4IaVzVeE13sKz2MymmtkkM3sCeBzo\nDmHwlVgbPkTSx5KWEOY4Q9JhkkZJWiRpvKSrJdVLHFTS+pJekLRQ0kRJvVNPnPo1WlJTSXdL+iUe\n92tJx8RBYB4GGiXV0PvFfepJul7SZEnzJX0i6cCU8xwk6Zt4zPcIU+BUi6TrJH0bX98ESTdI+t2U\nOpJOlfRT3G6wpBYp5SdL+irG9p2k8yX535lbjdd4C99CQu032fWEWXy/B+bFxPY48FfC3F5tCOPC\n1gcSiXQgsAlh+pkFwL+BtmWdNM4h9wphQsaTge+ArQjzg30InEeYzn2zuEui2eHhuO4vwGTgEGCI\npB3NbEycGHMwYfSuAYQZGW5J/+0o03ygNzAF2Jrw+hcDlyVt0xY4ATgCWAu4D3gIODy+5tOAfwHn\nAKOAbWKcSwFv5nCrmJkvBbIQkuPQpOc7EaZI/298vjdgQI+U/YYDl6Ws605IhiLUKA3YPal8E2A5\n0C9p3QTgovh4f2AF0L6MWHsBJSnrNov7tElZP5gwUSiEZP0d8Xb3uK5vjK9tOe/NMODOSryXZwDf\nJz3vF19vm6R1XeN5t4jPfwJOTDnOecBXpb1HvtTexWu8heegeNGqmFDTfYFQA0s2MuX5DsBOkv6W\ntK4IaEiYZqY9ISF+nCg0s4mSfi4nju0Ic4V9XYnYtyck+q9SJl2uT5h1mBjLR2aWPMjIiEqco1SS\njiYkyc2BxkCduCSbYmY/JT3/H/HDRdIcoDVwr6S7k7YpJrwm51byxFt4hhOmlV8K/GylXzibn/K8\nCLgSeLqUbadTtcRRlX2KCDXIHQnxJ1tYjeOWS9IuwH8I78H5wBxC88FNlThMoh03MV29c2XyxFt4\nFljlZ6D9FGhX1n6SviYklh2JSUVSG6BVBcdsKal9GbXeJfy+RvkZIbFuaGbvlHHcr4AekpRU692l\nnDjSsTuhNntVYoWkTUrZbiNJrc1sUny+E+F9+drMpkmaAmxmZoOqGY8rcJ54HYQLQkMlTQSeApYR\nLgztZGaXmNm3kl4lfI3uQ6h93sKqWmhp3iJ8FX9W0vmEdtnNgUZmNpjQ1tlA0v6EhLvAzL6T9Dgw\nUNKFhOS9DqFt+kcze45w0etC4FZJdwHbEmqZ6WgR5zBL9muMbSNJxxOaLQ4Ejitl/4XAI5IuIDTD\n3AO8ZGbjYnk/4I7Y7PAyoalne2AjM7s2zRhdLeDdXBxm9hphyvl9CO24HwOXEi4WJfQCxhPaWocQ\nZkueUM4xVwAHAx8AjwFfA7cB9WL5h4TE9SShOeOSuOvJhJ4NNwDfAEOBPYGJcb+fgKOAg4AxhKaB\nS9N8qccQknzycoGZDQFuBG4FPidcGLy8lP0nEJokhsT34ccYb+I1P0DoGXFijO09QrPP+DTjc7WE\nD4TunHNZ5jVe55zLMk+8zjmXZZ54nXMuyzzxOudclnnidc65LPPE65xzWeaJ1znnsswTr3POZdn/\nA9DfxXc7L+G8AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2a39291a748>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Confusion Matrix\n",
    "conf_mx = confusion_matrix(y_test, y_pred)\n",
    "plot_conf_matrix(conf_mx, train.author.unique(), title=\"MultiNB Pipe Confusion Matrix\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Parameter Tuning with grid search cv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 255,
   "metadata": {},
   "outputs": [],
   "source": [
    "# http://scikit-learn.org/stable/auto_examples/model_selection/grid_search_text_feature_extraction.html#sphx-glr-auto-examples-model-selection-grid-search-text-feature-extraction-py\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "\n",
    "parameters = {\n",
    "    'vect__max_df': (0.5, 0.75, 1.0),\n",
    "    'vect__max_features': (None, 5000, 10000, 50000),\n",
    "    'vect__ngram_range': ((1, 1), (1, 2)),\n",
    "    'tfidf__use_idf': (True, False),\n",
    "    'tfidf__norm': ('l1', 'l2'),\n",
    "    'clf__alpha': (1e-2, 1e-3),\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 256,
   "metadata": {},
   "outputs": [],
   "source": [
    "mnb_gs_clf = GridSearchCV(mnb_clf, parameters, n_jobs=-1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 257,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=None, error_score='raise',\n",
       "       estimator=Pipeline(memory=None,\n",
       "     steps=[('vect', CountVectorizer(analyzer='word', binary=False, decode_error='strict',\n",
       "        dtype=<class 'numpy.int64'>, encoding='utf-8', input='content',\n",
       "        lowercase=True, max_df=0.5, max_features=None, min_df=1,\n",
       "        ngram_range=(1, 2), preprocessor=None, stop_words=None,\n",
       "        strip...lse,\n",
       "         use_idf=False)), ('clf', MultinomialNB(alpha=0.01, class_prior=None, fit_prior=True))]),\n",
       "       fit_params=None, iid=True, n_jobs=-1,\n",
       "       param_grid={'vect__max_df': (0.5, 0.75, 1.0), 'vect__max_features': (None, 5000, 10000, 50000), 'vect__ngram_range': ((1, 1), (1, 2)), 'tfidf__use_idf': (True, False), 'tfidf__norm': ('l1', 'l2'), 'clf__alpha': (0.01, 0.001)},\n",
       "       pre_dispatch='2*n_jobs', refit=True, return_train_score='warn',\n",
       "       scoring=None, verbose=0)"
      ]
     },
     "execution_count": 257,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mnb_gs_clf.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 258,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.83594388450013624"
      ]
     },
     "execution_count": 258,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mnb_gs_clf.best_score_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 259,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "clf__alpha: 0.01\n",
      "tfidf__norm: 'l2'\n",
      "tfidf__use_idf: False\n",
      "vect__max_df: 0.5\n",
      "vect__max_features: None\n",
      "vect__ngram_range: (1, 2)\n"
     ]
    }
   ],
   "source": [
    "for param_name in sorted(parameters.keys()):\n",
    "    print(\"%s: %r\" % (param_name, mnb_gs_clf.best_params_[param_name]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 260,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = mnb_gs_clf.best_estimator_.predict(X_test)\n",
    "y_pred_prob = mnb_gs_clf.best_estimator_.predict_proba(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 261,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy score: 0.862717058223\n",
      "Log loss: 0.354775708031\n"
     ]
    }
   ],
   "source": [
    "print(\"Accuracy score:\", accuracy_score(y_test, y_pred))\n",
    "print(\"Log loss:\", log_loss(y_test, y_pred_prob))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 262,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV4AAAEqCAYAAABdmDjNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XeYVEXWx/Hvb0BQkoCAkjGDoKCA\nOWBCzKz6KgYUVNB1zWnNYs7rmhUVc95VEESMBAkGkLBgABFRBCQjGUbP+0fdgaadmW5merp7es7H\n5z5037rh3BZOV9etWyUzwznnXPrkZToA55yraDzxOudcmnnidc65NPPE65xzaeaJ1znn0swTr3PO\npZkn3iwgySTtUEz5FEmd0hhS7LlPl/RhJs5dGpJ2ljRe0jJJF5fiOE9KujGVsWWCpOWStst0HC7w\nxFsKkn6StFZSvbj1E6Jk2qIEx3xe0u2x68ystZkNi8r7RMf+v5h9KseeLzrG2ugf2zJJ4yQdVMw5\n+0haF22/RNJoSftE537FzDpv6nUkQ1KV6NzTJK2IPs9+JfncCnE1MMzMaprZwyU9iJmdb2a3pSCe\njcT8f7w4bv2l0fo+SR5nmKRzE21nZjXM7McShutSzBNv6c0ATi14I2lXYIsyPuci4FZJlYrZ5l4z\nqwFsCTwBvJ1g+zei7esDI6PtlbKIC/cf4DjgtCjOtsA44NAUHLs5MCUFxylLU4Gz4tadGa1PCUmV\nU3UslzqeeEvvJcI/lgJnAS/GbhBfK5HUQ9LI+ANJ6g2cDlwd1T4HRut/knRYzKZDgLXAGYmCM7M/\ngVeBusDWSWy/DngB2AbYKj7WglqapB8lLZB0n6S8mPKzJX0rabGkDyQ1L+w80fUcDhxvZl+ZWb6Z\nLTWzx8zs2WibRpLelbRI0g+SesXs30fSm5JejGr1UyR1iMo+BQ4GHo0+x52K+3+g4EFJ8yQtlTRJ\nUpuobKNfIJJ6RbEsimJrFPfZnB/V4BdLeizBl9dXQDVJraP9WxO+tL+KOWYdSYMkzY+OOUhSk6js\nDuCAmOt8NCaOf0iaBkyLWbdD9CtjgqSLovWVJI2SdFMxcboU88Rbep8DtSS1imqUpwAvl+RAZtYX\neIWotmpmxxa1KXAjcLOkzYo7ZhTTmYSa+W+JYpBUFegBzDKzBUVs9jegA7AHcDxwdrRvV+A64ARC\nzfkz4LUijnEY8KWZ/VJMOK8Bs4BGwEnAnZJia8PHAa8DtYF3gUcBzOyQ6NwXRp9johpkZ+BAYKfo\nWKcAC+M3knQIcBdwMtAQmBmdP9YxQEdC7f1k4IgE54794v7Llzbh3+hzhBp8M2BVzHVeH3edF8bs\n1xXYC9gl9mBmVvCFfaukVsA1QCXgjgRxuhTyxJsaBf94Dge+A34t6xOa2bvAfKCo9r0rJS0BVgD/\nBm40sz+KOeTJ0fa/AO0J/3CLco+ZLTKzn6NjFzS1nAfcZWbfmlk+cCfQroha71bAnKJOIKkpsD/w\nTzNbbWYTgGeA7jGbjTSzwdF1vURIdiWxDqgJtAQUxV9YbKcD/czsazNbA1wL7BPXJn23mS2JPpuh\nQLsE534ZODX6Au1G3Je2mS00s/+a2UozW0ZIkEW218e4K/p/tCq+wMwmA7cD7wBXAt0T/N1wKeaJ\nNzVeIrRT9uCvNZaydANwPbB5IWX3m1ltwk/XDsB9ko4s5lhvmlltM2tgZoeY2bhito2tpc4k1Egh\n1Moeim7QLSG0RQtoXMgxFhJqjUVpBCyKkk3suWKPNTfm9Upg85K0aZrZp4Ra5GPAb5L6SqpVREwz\nY/ZbTriO4mKqkeDcPwM/EL6kpsX/ApBUTdJTkmZK+h0YAdRO0F4PG/8/KswLQAtgsJlNS7CtSzFP\nvClgZjMJP+WPAt4uZJMVQLWY99sUd7hNOO9HhH+0FxSzjUU1nFHA0ckeO4GmMa+bAbOj178A50UJ\nvGDZwsxGF3KMj4E9C9orCzEbqCupZty5Svprotj/B2b2sJm1B1oTmhyuKiKm9bV3SdUJNffS/sJ5\nEbiCwr+0rwB2BvYys1qEJhEIX2hQ9N+XRH+PHgcGAUdI2n/TwnWl5Yk3dc4BDjGzFYWUTQBOiGov\nO0TbFuU3YFP6W15P6DpVJEktCT/bU3WX/6ropk9T4BLgjWj9k8C1MTeLtlRMt7dYZvYx8BHwjqT2\nCl3iakY3p86Oan6jgbskbS5pN8Ln9koJYy7y/4GkjpL2in7urwBWA4X99H4V6CmpXdQWfifwhZn9\nVMKYCrxBaGd+s5CymoR23SWS6gI3x5Vv6t8XJHUnNCf1AC4GXpBUbM3cpZYn3hQxs+lmNraI4gcJ\nvRB+I/zEKy55PAvsEv1c75/EeUcBXxZSVNAzYgXwIeEGzVOJjpekAYRuXxOA96KYMbN3gHuA16Of\nxZOB4po3TgIGExLP0mj7DoTaMIS24xaEmuY7wM1RLb8kivt/UAt4GlhMaEpYCNwffwAz+4RwU/O/\nhPbp7QntsqViZqvM7OPC2mMJbehbAAsIN3KHxJU/BJwU9XhI2F9ZUrPomGea2XIzexUYS/h8XJrI\nB0J3m0KSATua2Q+ZjsW58sprvM45l2aeeJ1zLs28qcE559LMa7zOOZdmFXYADVXewlSlZuINK6C2\nrZplOoSs5b8Qi/bLzzNZuGBBSgdWqlSruVl+YZ09Nmar5n9gZl1See6yVHETb5WaVN355EyHkZWG\njyrxKIo5b23+n5kOIWsdduBeKT+m5a9K6t/p6gmP1Uu4URapsInXOVceCJR7LaKeeJ1z2UtAXqJh\nKcofT7zOuexW5uPxp58nXudcFvOmBuecSz+v8TrnXBoJr/E651x6yW+uOedc2nlTg3POpZPfXHPO\nufQSXuN1zrm08xqvc86lk6BS7t1cy72vEudc7ijoTpZoSeZQ0oWSxkpaI+n5uLJqkh6XtEDSUkkj\nYsok6R5JC6PlXmlD+0c0+ek4SSujP9slisUTr3Muu0mJl+TMBm4H+hVS1heoC7SK/rwspqw30BVo\nC+wGHAOcF0JTFcLkry8DdQgTqQ6I1hfJE69zLospZTVeM3vbzPoTZpHecAZpZ+A4oLeZzTezP8xs\nXMwmZwEPmNksM/sVeADoEZV1IjTZ/tvM1pjZwyFoDikuFk+8zrnsllyNt17UjFCw9N6EM+wFzARu\niZoa/ifpxJjy1sDEmPcTo3UFZZNs4xHyJ8WUF8pvrjnnspeSfnJtgZl1KOFZmgBtgP8CjYB9gPck\nfWNm3wI1gKUx2y8FakTtvPFlBeXFTm/jNV7nXHZLUVNDMVYB64DbzWytmQ0HhgKdo/LlQK2Y7WsB\ny6NabnxZQfmy4k7oidc5l91Sd3OtKJMSlE8h3Fgr0DZaV1C2W2wvB8INuCkUwxOvcy6Lpe7mmqTK\nkjYHKgGVJG0uqTIwAvgZuDbaZj/CTbMPol1fBC6X1FhSI+AK4PmobBjwB3CxpKqSLozWf1pcLJ54\nnXPZLXU13hsIzQrXAGdEr28ws3XA8cBRhPbZp4Ezzey7aL+ngIHA/4DJwHvROsxsLaGr2ZnAEuBs\noGu0vkh+c805l70kyEtNmjKzPkCfIsqmEG6qFVZmwNXRUlj5eKD9psTiidc5l918kBznnEszHyTH\nOefSzGu8zjmXRvKB0J1zLu2Ul3uJN/euKEs8efPpzPzkLsa+dd36dS/d3ZPPX7+Gz1+/hu/eu4XP\nX78GgEP2asmoV67mqzevY9QrV3NQx53W73NS5z348o1rGfef67njkuPTfh1l7R/nncP2zbZh7/a7\nrV+3aNEijj+6M7u32Znjj+7M4sWLATAzrr78Etq13ol9O7ZjwvivMxV2Wlz893NptW0jDthzwyiD\n9955K7vu1JxO+7an077t+eiD9zfaZ9YvP9N8m9o89tC/0h1umQgTUCjhUt544i0jLw38nOP/8dhG\n67pf8xx7d7ubvbvdTf9PJjDg0wkALFyynJMufYqOJ99Jr5teot/tZwJQd8vq3HlpV446/xHan3QH\nDbaqRac9d/rLucqz07qfxX8HDN5o3YP338NBnQ5l/OTvOajToTx4/z0AfPTB+0yfPo3xk7/noUef\n5PKL/5GJkNOm2+ln8fo7g/6y/vx/XMKw0eMYNnochx9x5EZlN1xzJYce3iVdIZY9JbmUM554y8io\nr6ezaOnKIstPPHwP3hwSRp6b+P0s5swP42x8M30OVatsRpXNKrNt462Y9vM8FixeDsCnX3xH10MT\njrFcruy3/4HUqVt3o3WDB73LaWeEL5/TzjiT9wYOAOC9Qe9y6mndkUTHvfZm6dIlzJ0zJ+0xp8u+\n+x9AnTp1E28YGTxwAC1abEvLVruUYVTplri26zVel5T99tie3xYtY/rP8/9S9rfD2jHx+19Yuy6f\n6b/MZ+cWW9OsYV0qVcrjuIPb0mTrOhmIOL3mz/uNbRo2BGCbhg2ZP38eAHNm/0rjJk3Xb9eocRNm\nz/41IzFm0rN9H+egvXfn4r+fy5KoGWbFihU88uB9XHntjRmOLvU88ZYRST9JWiVpeczyaEx5J0km\n6eq4/VpE6wv2+UnSNem/gk1zcpcOvDVk7F/Wt9puG26/+HguvP11AJYsW8XFd77By/eczSf9LmPm\n7IX88cef6Q43a2w85GlQHv/RlUaPc8/jq0nfM3T0OLbepiE3XXcVAPfecQvnXXgJNWrUyHCEqZeX\nl5dwKW+yqVfDsWb2cRFlZwGLoj/vLaS8tpnlS9oH+ETSBDMbUlaBlkalSnkcf0hb9jtt48to3KA2\nb/yrN+fe+BIzZi1Yv37wiMkMHjEZgLNP2K9CJN76DbZm7pw5bNOwIXPnzKF+/QZAqOH+OuuX9dvN\n/nUWDRs2ylSYGdGgwdbrX3fvcQ6n/19XAMaN/ZKBA97m1huvZenSJeTl5VF186qce145bwcvp224\niWT9V4WkasBJwD+AHSUVOdixmY0hDMfWJk3hbbJD9tqZqT/9xq/zlqxft2WNLXj7kfO56ZF3GTPx\nx422r18n1GBq19yC3icfwHPvjElrvJlw5NHH8urLLwLw6ssvctQxxwFw1NHH8tqrL2FmfPXF59Sq\nteX6JomKYu7cDW3agwf2p+UuYaKDQR8O4+spP/D1lB8474KLufSKa8p/0gWUo2282VTjLcqJhMGG\n3wJOJ4wC9Jff6dF4mPsSptwYX9iBoulAwpQgm5XtT7IX7urBAe13pF7tGvww5DZue3IwL/Qfw/8d\n0X79TbUC53c7kO2b1ueaXl24ple4I33s3x9l/uLl3H/1Sey6U2MA7uo7hB9+nlemcafb2WeexsjP\nhrNwwQJabd+Ma2+8mcuv/CdnndGNl17oR5OmzXjhlTcA6NzlKD784H3atd6JatWq8dhTz2Y4+rLV\nu+cZjPpsOIsWLmC3nVtw9XU3MXrkcCZPmogkmjZrwf0PP57pMMtceUysiaiwdrO0ByH9BNQD8mNW\nX2VmT0v6GJhsZpdKOhV4GGhkZusktQBmEIZyM2Au8EQ04Vyx8qo1sKo7n5zaC8kRv41J+PFVWGvz\nc7+pp6QOO3AvJnw9LqVZsvJW21mto25PuN3il08fV4qpf9Ium2q8XePbeCU1BQ4Gro1WDSBMw3w0\n0D9m03pmFpu0nXO5QKC83KvxZnsbb3dCjAMlzQV+BDYnNDc45yqAVLXxSrowmoF4jaTni9jm5qin\n1GEx66pK6ifpd0lzJV0et8+hkr6TtFLSUEnNE8WS7Yn3TOAWoF3MciJwtKStMhmYc67spfjm2mzg\ndqBfoeeStifcyI9/KqcPsCPQnPAL/GpJXaJ96gFvAzcCdQn3n95IFEg2Jd6Bcf14hwItgMfMbG7M\n8i7wA3BqRqN1zqVFqhKvmb1tZv2BhUVs8ijwTyB+2p4zgdvMbHE03fvTQI+o7ARgipm9ZWarCUm6\nraSWxcWSFW28ZtZiE7dvHfM29xqAnHMbJPcvvJ6k2N5Ofc2sb9KnkP4PWGtmg2MTuaQ6QCNgYszm\nEwnzrEHoRbW+zMxWSJoerf+OImRF4nXOuUKJZJ9MW1DSXg2SagB3Ap0LKS7od7o0Zt1SoGZMefyz\n/7HlhfLE65zLamnox3sL8JKZzSikbHn0Zy1gdczrZTHlteL2iS0vVDa18Trn3EbS9OTaocDFUY+F\nuUBT4E1J/zSzxYSbbW1jtm9LeEKW6M/1ZZKqA9vHlBfKE69zLrulaDxeSZUlbQ5UAipJ2lxSZULi\nbcOGnlOzgfOAggG1XwRukFQnumnWC3g+KnsHaCPpxOjYNwGTzKzI9l3wxOucy2ZK6bCQNwCrgGuA\nM6LXN5jZwtieU8AfwGIzK2hmuBmYDswEhgP3FQzCZWbzCV1c7wAWA3sB3RIF4m28zrmslqphH82s\nD6G7V6LtWsS9XwOcHS2Fbf8xUGz3sXieeJ1z2S0HO4x64nXOZbVcHJ3ME69zLmuV1/F2E/HE65zL\nap54nXMuzXJxWEhPvM65rOY1XuecSyd54nXOubQSkIN51xOvcy6bea8G55xLuzy/ueacc2kkb2pw\nzrm0El7jdc65tPMar3POpZnfXHPOuTSScrOpwQdCd85lsdRN/SPpQkljJa2R9HzM+r0lfSRpkaT5\nkt6S1DCmXJLukbQwWu5VzEkltZM0TtLK6M92iWLxxOucy2pS4iVJs4HbgX5x6+sAfYEWQHPCRJXP\nxZT3Jkzn3hbYDTiGMDUQkqoAA4CXo+O8AAyI1hfJE69zLqulqsZrZm+bWX9gYdz6983sLTP73cxW\nAo8C+8VschbwgJnNMrNfgQeAHlFZJ0KT7b/NbI2ZPUzojHFIcbF44nXOZa8kartlcO/tQDaeJbg1\nMDHm/cRoXUHZJDOzmPJJMeWF8ptrzrmstQn9eOtJGhvzvq+Z9d3k80m7EWYKPj5mdQ1gacz7pUCN\nqJ03vqygvGZx5/HE65zLakk2JSwwsw6lPM8OwPvAJWb2WUzRcqBWzPtawHIzM0nxZQXly4o7lzc1\nOOeyWjqaGiQ1Bz4GbjOzl+KKpxBurBVoy4amiCnAbtr422E3Nm6q+IsKW+PdrWVTPhz+YKbDyEqH\nPjAi0yFkreFXHZTpELJWXlk86JDC8XglVSbkvEpAJUmbA/nA1sCnwGNm9mQhu74IXC5pMGDAFcAj\nUdkw4A/gYklPAr2i9Z8WF0uRiVdStWQvKLoT6JxzKZXi8XhvAG6OeX8GcAshmW4H3CxpfbmZ1Yhe\nPhWV/y96/0y0DjNbK6lrtO5u4Fugq5mtLS6Q4mq8y6OAklEpye2cc24TKGVPrplZH6BPEcW3FLOf\nAVdHS2Hl44H2mxJLcYn3yE05kHPOlYUKNVaDmX2QzkCcc+4vcnQ83qR7NUjaKnrW+UFJW0XrOkpq\nWnbhOecqstDGm5on17JJUr0aJLUFPgHmA9sT7ugtBI4jPN/cvYzic85VcOUxsSaSbI33AeAZM2sF\nrIlZ/z5wQMqjcs65SF6eEi7lTbL9eDsSjcYT51dgm9SF45xzMXK0jTfZxLuGvz4WB7ATsCB14Tjn\n3AbK0endk21qGAjcED35AWCSGhM6DL9TJpE55xwZGZ2szCWbeK8EmgBzgS0Ij8P9CKwFriub0Jxz\nLjyKnGgpb5JqajCzxZL2AboAexAS9tfAYDP7swzjc85VYLk651rSg+RECXZwtDjnXFrkYN7dpAco\nukj6UNKsaPlIkj9W7JwrU7n4AEVSiVfSRcC7wG/AndEyF+gv6eKyC885V9Hl4s21ZJsargEuM7PH\nYtY9LmkMYai1h1MemXOuwhOhS1muSbapYUugsEFzPqTw/r3OOVd6EpXyEi/lzab04z2+kPXH4zfb\nnHNlKBebGopMvJIuKFgII6/fIKm/pGuipT+hmWF8uoJ1zlUsInX9eKPRFcdKWiPp+biyQyV9J2ml\npKHRHGwFZVUl9ZP0u6S5ki5Pdt+iFNfGe2Pc+9XAXtESu+4i4K5EJ3LOuZJIYY12NnA7cAThQbDo\n+KoHvA2cS/h1fxvwBrB3tEkfYEegOWFsmqGSvjGzIUnsW6jiBkJvWIILc865lEpVdzEzezs6XgfC\nk7gFTgCmmNlbUXkfYIGklmb2HXAm0NPMFgOLJT0N9ACGJLFvoXx6d+dc1pJI9uZavagZoWDpvQmn\naQ1MLHhjZiuA6UBrSXWARrHl0evWifYt7oRJP7kmqQUhuzcDqsSWmdkFyR7HOec2RZL13QVm1qGE\np6hBmOQh1lKgZlRW8D6+LNG+RUp2BorDCQ9QfMeGDL8dYXbhL5M5hnPOlUQankxbzl+7xdYClkVl\nBe9Xx5Ul2rdIyTY13AncbWa7E8bmPYVQ8x1OSMjOOZdyoVdD4qWUpgBt159Tqk6Y4mxK1K47J7Y8\nej0l0b7FnTDZxNsSeDl6nQ9sEbVl3ARcleQxnHNu0yQxTkOyNWJJlSVtTvilXknS5tEY4+8AbSSd\nGJXfBEyKuTn2IqE7bR1JLYFewPNRWaJ9C5Vs4l3BhnbdOYSMDmDAVkkewznnNlkK51y7AVhFGALh\njOj1DWY2HzgRuANYTOgy2y1mv5sJN8xmEn7l32dmQwCS2LdQyd5c+xLYF/iW0IXiPkmtohN6G69z\nrkwUNDWkgpn1IfTJLazsY8Iv+8LK1gBnR8sm7VuUZGu8VwETotc3A6OBc4B50Z+uGJdc0ItdtmvM\ngXu122j9M08+xr57tObAPdty643XrF8/ZfIkjjr0AA7csy0H7b07q1evjj9kuXbjMTvzwaX78nqv\njuvXHdqyPm/07sgX1x1Eq4Ybbgh3ad2AV87tsH754rqD2GnrcKP58Fb1efXcDrzRuyMXHbJd2q8j\n3ZYsWcLp3f6P3XdtxR677cIXn48B4InHHqFdm5Z0aNeG66+9OsNRpl4uDguZ7AwU38e8Xgb0BJC0\nGdC0bELLHd1OP5Nzel/Ahef1XL9u5IhhDBk8kKFjvqZq1arMnz8PgPz8fP7RqweP9X2O1ru2ZdHC\nhWy22WaZCr1MDJo4lzfH/sotx7Zav276/BVc/Z/JXHvUzhttO2TKPIZMCZ/N9vWr88D/tWHqb8vZ\ncovKXHzo9nTvN44lK9dx87Et6diiNl/9tCSt15JOV11xKYd3PoJXXn+LtWvXsnLlSoYPG8qgge/y\nxbiJVK1alXnz5mU6zJQrf2k1sdI+QLELMC0VgeSyffY7gNp16my07oVnn+Kiy66iatWqANSv3wCA\nYZ98xC6td6X1ruFGad2ttqJSpUrpDbiMjf9lKb+vyt9o3U8LVzJz0api9zuidQM++CYklsa1t+Dn\nRatYsnIdAF/OWMwhLeuXTcBZ4Pfff2fUZyM4q2f4gVmlShVq167NM32f5Iqr/rn+71GDBg0yGWbK\nSbk555o/uZYh03+YxhejR9Ll4P3oeuShjB83dv16SZzS9WgOO2BPHv33/RmONHscvksDPoxqv78s\nXkXzrarRcMvNqSTRaed6bF2raoYjLDszZvxIvfr1Oa/X2eyz5x5ccP65rFixgmnTpjJ61GcctP/e\nHHFYJ8aN/SrToaZcCm+uZY2sSLySfpJ0WNy6HpJGxpSvkrRc0m+SnpNUIyobJuncTMRdGvn5+SxZ\nsoT3Px3JTbfdTa8ep2Fm/PFHPl98PprHn32Bdz8YxuCBAxgx7NNMh5txrRvVZPW6P5g+fwUAy1bn\nc8+Qqdz5t13oe2Y75ixZTf6fluEoy84f+flMGP81vXqfz5gvv6Zateo8cN/d4e/R4sUM+2wMd9x1\nL91POwWz3PocKtSwkFnoWDOrQZjluCOha0i51ahRE44+riuS2KNDR/KUx8KFC2jYqDH77ncAW21V\nj2rVqnFY5y78b6KPvNl5lwZ8MGXj9svPpi2k5/Nfc84L45m5aCW/JGiqKM8aNW5C4yZN6LhnGBzw\nbyecxITx42ncuAnHdT0BSXTouCd5eXksWLAgw9GmjkjczJBzTQ2SdiluYUN/3rQxs1+B94E26T53\nKh15zHGMHD4UgOnTprJu3Vq22qoeBx/amW+m/I+VK1eSn5/P6FGfsdPOrRIcLbcJOLRVAz76ZuPE\nW6dauOlYc/PKnNS+MQMmzMlAdOmxzTbb0KRJU6Z+H+5zDxv6CS1bteLY445nePSLaNrUqaxdt5Z6\n9eplMtTUSqK2Ww7zbsJeDZMJD0kURQnKU05SU+AowhiYm7pvb6A3QJOmzVIcWdHO63kGo0eOYNHC\nBbRruS1XXXcTp3bvwaUX9OLAvdpRpUoVHn7yWSRRu04dzv/HJXTptA9IHNa5C4d3OSptsabD7V1b\n0b55bWpvsRmDLtqHviNm8PvqfK7svCN1qm3GgyfvytTflnPx65MA2L1ZbeYtW8OvSzbuVndF5x3Y\nsUHoWvbMyJn8nMM1XoD7H3yYs3ucwdq1a9l22+148ul+VK9enfN7n0OH3XelSpUq9H3m+XLZvao4\nuXY9ACquPUjSzkUWxojtblaiIKSfgHqEx5ELVAG+NrP948qXAu8BV5jZKknDgJfN7JlNOWe7Pdrb\nh8M/L03YOevoh0dlOoSsNfyqgzIdQtbaf5+OfD1ubEqz5NY7tLFT7v9Pwu0e+VurcaUYnSztiq3x\nljahbqKu0RMgQLi5RhjVvdBy51zFUA47LSSU9Hi8zjmXCZ54s1fBqEMF/jCzdRmLxjmXEuHmWe5l\n3vLUnaw4TxBGGipYnstsOM65VEnDeLxplxU1XjNrUci654nGvCysPGa7TmUTlXMu0wQFc6rllE2q\n8UqqIaltNDiOc86VubwklvImqZglVZf0IvA7MI5oRDJJj0q6vgzjc85VcKl6gEJSC0mDJS2WNDfK\nX5WjsnaSxklaGf3ZLmY/SbpH0sJouVelbHhO9sviLsJAv/uyYcI3gA+B/ytNAM45VxQl8bjwJjwy\n/DhhDPGGQDvgIOACSVWAAYTpzeoALwADovUQHrrqSphbbTfgGOC80lxXson3eOAiM/ucjZ9U+4Yw\n27BzzpWJFD4yvC3wppmtNrO5hNl0WgOdCPe7/m1ma8zsYULz8iHRfmcBD5jZrGjIggeAHqW5pmQT\nb33CN0W86qU5uXPOFUdA5TwlXIB6ksbGLL0LOdxDQDdJ1SQ1Bo5kQ/KdZBs/xjspWk/058SYsokx\nZSWSbOIdRxgfoUBBgGcDY0oRM56LAAAb4klEQVQTgHPOFSfJGu8CM+sQs/Qt5FDDCQnzd2AWMBbo\nD9QgDEUQaylQMAdVfPlSoEZp2nmT7U52PTA4mtq4MvAPSQVVdH943TlXNlLUT1dSHvAB8BThXlUN\noB9wD2Hm9Fpxu9QClkWvl8eV1wKWWykGPk6qxmtmIwgJtgHwK3ACYcr3/czMZxl2zpUZJfFfEuoS\nemM9GrXjLiQ8aHUUMAXYLa4Gu1u0nujPtjFlbWPKSiTpByjMbBxwSmlO5pxzmyJV07ub2QJJM4C/\nS7qfUOM9i9BeOwz4A7hY0pNAr2i3gqlfXgQulzSY0Mx6BfBIaeJJth9vteKW0gTgnHPFqZSnhEuS\nTgC6APOBHwjDzF5mZmsJ3cXOBJYQ7l11jdZDaJ4YCPyPMEb5e9G6Eku2xruc4gc8z61pcJ1zWSFV\nNV4AM5tAuC9VWNl4oH0RZQZcHS0pkWziPTLu/WbA7oTxcm9MVTDOObeRcjq1TyJJJV4z+6CQ1YMk\nTQXOILSBOOdcypXHySwTKe34EmPZ8HSHc86lVEFTgw8LGYmeY/4HoXuZc86VAVEpB2u8SSVeSfPZ\n+OaagNrAWsKdQOecSzlRgdt4gRvi3v9J6JIx2swKG8PBOedKr5w2JSSSMPFG41WuAwZHI/o451za\nVMiba2aWDzwKVC37cJxzboOCpoYUDQuZNZJtaviS8HzyzDKMxTnn/iIX51xLNvE+CjwgqRFhiMgV\nsYVm9k2qA3POOVE+51RLJNnE+2b05+PRnwU9HBS99keGnXOppzD9T65JNvG2KtMonHOuCLmXdhMk\nXkn9gEvM7Ps0xeOcc+uFJ9dyL/Umaj45C9giHYE451xhKuIjw+XwkpxzuUMVto23xPMKOedcaeRq\nr4ZkrmmupD+KW8o8SudchSUp4bIJx+om6VtJKyRNl3RAtP5QSd9JWilpqKTmMftUldRP0u+S5kq6\nvLTXlEyNtzdhOgznnEu7VDU0SDqcMKvwKYSHwhpG6+sBbxMmdhgI3Aa8Aewd7doH2BFoDmwDDJX0\njZkNKWksySTegbk6EE4uPhGTCp/9s1OmQ8hara56L9MhZK3Zs5am/JgSqRwW8hbgVjP7PHr/aziH\negNTzOyt6H0fYIGklmb2HWEExp5mthhYLOlpoAdQ4sSbqKnB23edcxmVZFNDPUljY5becceoBHQA\n6kv6QdIsSY9K2gJoTZhtGAAzWwFMB1pLqgM0ii2PXrcuzTV5rwbnXFZLMgktMLMOxZRvTZgr8iTg\nAMKIiwMIQ97WIAxzG2spUDMqK3gfX1ZixdZ4zSwvV5sZnHPlQ4pGJ1sV/fmImc0xswXAv4CjCLOo\n14rbvhawLCojrrygrMRysaeGcy5HhO5kSrgkErXPzqLw5tMphNEXwzml6sD2hHbfxcCc2PLo9ZSS\nX5UnXudcVhN5Srwk6TngIkkNorbbS4FBwDtAG0knStocuAmYFN1YgzCL+g2S6khqCfQCni/NVXni\ndc5ltRQOhH4b8BUwFfgWGA/cYWbzgROBO4DFwF5At5j9bibcbJsJDAfuK01XMijFLMPOOVfWCpoa\nUsHM1gEXREt82cdAyyL2WwOcHS0p4YnXOZe9yunUPol44nXOZTVPvM45l0YipU+uZQ1PvM65rKYc\nfI7LE69zLqvlYIXXE69zLrt5jdc559IozLmW6ShSzxOvcy57bdqTaeWGJ17nXFbLvbTridc5l8Vy\ndXp3T7zOuayWe2nXE69zLtvlYOb1xOucy2re1OCcc2mWe2nXE69zLtvlYOb1gdCdc1lLhCfXEv23\nSceUdpS0WtLLMetOkzRT0gpJ/SXVjSmrK+mdqGympNNKe12eeJ1z2SuJ2SdK0AT8GGEminAKqTXw\nFNCdMBvxSuDxuO3XRmWnA09E+5SYNzU457JaKu+tSeoGLAFGAztEq08HBprZiGibG4FvJdUE/iRM\nC9TGzJYDIyW9S0jS15Q0Dq/xOueyWDINDQKoJ2lszNL7L0eSagG3AlfEFbUGJha8MbPphBruTtHy\nh5lNjdl+YrRPiXmN1zmX1ZKs8S4wsw4JtrkNeNbMftHGB60BLI3bdilQE/ijmLIS88SbBhf//Vw+\nfH8w9eo3YORXEwC469abef+9d8nLy6Ne/QY88tSzNGzYCICRI4Zzwz8vZ926fOputRUDP/g0k+Gn\n1c47tKBmjZpUqlSJypUrM+qLsSxatIjup53CzJk/0bx5C15+7U3q1KmT6VDLxD3dduOQXRqwcPla\nutw7AoDLj9yJw9tszZ9mLFy+litfnci839ewXYPq3HdqW1o3qcUD703l6WE/rj/OZzcezPLV+fxp\nRv6fxvH/GpWpSyoVkZpODZLaAYcBuxdSvByoFbeuFrCM0NRQVFmJeVNDGnQ7/Sze6D9oo3UXXnoF\nI74Yz7Ax4+jc5Sjuv+t2AJYuWcLVl13Ey2++w6ixE+n30uuZCDmjhnw8lC/GTWDUF2MBuP/eu+l0\nyKFM/nYanQ45lPvvvTvDEZad/345ix59v9xoXd9Pf+TI+z7j6PtH8umUeVx8xI4ALF25jlvensIz\nQ2cUeqzTHv+co+8fWW6T7npKYkmsE9AC+FnSXOBK4ERJXwNTgLbrTydtB1QlTAM/FagsaceYY7WN\n9ikxT7xpsO/+B1CnTt2N1tWsteFLdOXKlRT89Pnvm69xzHFdadK0GQD1GzRIX6BZatDAAZzR/SwA\nzuh+FgPf7Z/hiMrOlz8uYsmKdRutW74mf/3rLapUwiy8Xrh8LZN+Wcq6P/5MZ4hplxcNDVnckoS+\nwPZAu2h5EngPOAJ4BThW0gGSqhPagd82s2VmtgJ4G7hVUnVJ+wHHAy+V5pq8qSGD7uhzI2+89jK1\nam1J/8EfATD9h2msW7eO47ocyvLlyzjvgos45bTuGY40fSRx7JGdkcQ5vc7jnF69mffbbzRs2BCA\nhg0bMn/evAxHmX5XHrUzf+vQmGWr8zntsc8Tbm8GL56/F2bw2piZvDbmlzREWTZS0dRgZisJ3cTC\nMaXlwGozmw/Ml3Q+IQFvBXwM9IzZ/QKgHzAPWAj83czKR41X0k+S1kqqF7d+giSTtI+k3yVViil7\nuoh1T0avW0v6UNJiSUskjZN0VLquqbSu73Mbk76fwUmnnMozT4Vug/n5+Uyc8DWv/fdd3uo/mPvv\nuZMfpk1NcKTc8enwUYz56mv6D3qfp554jJGfjch0SFnh/sHfs9+tnzJg3K+ceUDzhNuf9PBojn1g\nJD37fkn3/Vqw53Z1E+6TlZJpZihBZjazPmZ2Rsz7V82smZlVN7PjzWxRTNkiM+salTUzs1dLdU2k\nv6lhBnBqwRtJuwJbRG8XAJWAPWK2PwCYHbfuQKDgX+NA4CNCx+YGwMXA72UReFk68eRuDBrwDgCN\nGjfhkMOOoHr16mxVrx777rc/U/43KcMRpk+jRuEGY4MGDTiu69/46qsvabD11syZMweAOXPmVOjm\nl3e/nk2X3Rom3G7e72uA0Bzxwf/m0rZZ7bIOrcyk+sm1bJDuxPsScGbM+7OAF6PX64DPCYkVSQ2A\nKsAbcet2AkZENedtgafNbG20jDKzkWm5klKa/sO09a+HvDeQHXfaGYAjjz6Wz0ePJD8/n5UrVzLu\nq6/YaeeWmQozrVasWMGyZcvWv/74ow9p3boNRx9zHC+/9AIAL7/0Ascce3wmw0y7FvWqrX99WJut\n+XHe8mK336JKJapXrbT+9QE71+f7uaW6CZ8xokyeXMu4dLfxfg50l9SKcLfwFGB/4PaofAQhyT4Q\n/TkyWi6MWTfDzGYp3I36AXhZ0jPAGDP7rbiTR52qewPrb16lQ68eZzDqs+EsWriAXXdqwT+vv4mP\nPxjCD9OmkpcnmjRrzgMPPQbATi1bccjhR3DgXnuQl5fHGT160qp1m7TFmknzfvuNU076GwD5f+Rz\nSrfT6HxEF9p36MgZp57MC889S9OmzXjl9bcyHGnZeah7O/beYSvqVK/C6JsP4d9DptGpVX22a1AD\nM+PXxau4/q3/AVCvZlXevXw/amxeGTPoeVALOt89gjo1NuOpnqFLa6VK4t1xsxnx3fxMXlaplMfE\nmois4BZpWZ9I+gk4F9gbqA4MJzxBciShtrstobvHf4F6wL+BycBrwHRgm2hdLTPrGR2zCeGxvSOj\n/UcC55jZhupkEdrt0d4++eyLlF1fLqle1e+5FqXVVe9lOoSsNfvVS1nz27SUpsk2bfew/wxJ/CO2\nVaPq45J4gCJrZKI72UvAaUAPNjQzFPic8BRJG0Lt9rPo+ehfYtatv9tiZrPM7EIz2x5oDqwo5JjO\nuXIsF5sa0p54zWwm4SbbUYT+cbFlqwmjBh0DNDSz76Kiz6J1uxGTeOP2/YUwilDF+F3uXAVRBp0a\nMi5TD1CcAxwSdU6ONwK4lDB6UIGR0bq50QAWSKoj6RZJO0jKi262nU2oNTvnckUOZt6MJF4zm25m\nY4soHk7oGhbbsDMyWhdb211LaBP+mNCFbDKwhtCE4ZzLAVLKnlzLKmm7i2JmLYpYn0/Md5aZfUDc\nd1jUWyF+3QpCdzTnXA4rf2k1Mb997ZzLbjmYeT3xOueyWPl8Mi0RT7zOuaxWDptwE/LE65zLWgWP\nDOcaT7zOuazmTQ3OOZdmuVjj9RkonHNZLRXPT0iqKulZSTMlLZM0XtKRMeWHSvpO0kpJQyU1j9u3\nXzQ2+FxJl5f2mjzxOueyl8KsJImWJFQmjPlyELAlcCPwpqQW0VOvb0fr6gJjCcPRFugD7EgYD+Zg\n4GpJXUpzWd7U4JzLWqm6uRY9cNUnZtUgSTOA9oTpfqaY2VsAkvoACyS1jMaLORPoaWaLgcWSniY8\nITukpPF4jdc5l9XKYqgGSVsTJlWYArQGJhaURUl6OtBaUh2gUWx59Lp1CU67ntd4nXNZLckabz1J\nseO/9DWzvoUfT5sRJrZ8wcy+k1QDiB8pfilQkzBMbcH7+LIS88TrnMtqSXYnW5DMQOiS8ghjgq8l\nzGwDsByoFbdpLWBZVFbwfnVcWYl5U4NzLqulaiD0aLqwZwmT455oZuuioilA25jtqgPbE9p9FwNz\nYsuj1+VjenfnnNtUySTdTbj59gTQCjjWzFbFrH8HaCPpREmbAzcBk2ImYngRuCEaA7wl0At4vjTX\n5YnXOZfVUjG9e9Qv9zygHTBX0vJoOd3M5gMnAncAi4G9gG4xu99MuNk2kzBe+H1mVuIeDeBtvM65\nbJea7mQzizuSmX0MtCyibA1hdpuzSx9J4InXOZfVcvCJYU+8zrlsVj6n9knEE69zLmvl6rCQfnPN\nOefSzGu8zrmslos1Xk+8zrms5gOhO+dcGkmQl3t51xOvcy7LeeJ1zrn08qYG55xLM7+55pxzaZaD\nedcTr3MuuyU5p1q54onXOZe1cvXJNZlZpmPICEnzCcO8ZYN6wIJMB5Gl/LMpXjZ9Ps3NrH4qDyhp\nCOEaE1lgZqWa+TedKmzizSaSxiYzbUlF5J9N8fzzKZ98rAbnnEszT7zOOZdmnnizQ6HTUDvAP5tE\n/PMph7yN1znn0sxrvM45l2aeeJ1zLs088TrnXJp54s1CysVnJJ1z63nizULmdzxdCUk6VlLLTMfh\niudjNWQRSb2AbYAvgLFmtijDIblyRNIbQH3gDkkzzGxNpmNyhfPuZFlCUn+gMbAcaAA8ZGZ9Jclr\nwC6RKOk2Ao4GVpnZurjyPDP7MyPBub/wpoYsIGkgsIWZdTSzg4HhwHmSKlf0pCvpJEmtMx1HNpPU\nAWgGdDGz381snaT6kg6SdDiAmf0pyf+9ZwlvasgwSQcTaimdYlYPINR+qwNLMxBWVpDUBrgNeFfS\ns2Y2NdMxZSkDaprZCkkNgD2AF4HFQB1JI8zsJK/xZg//BswwMxsK3Ax8IKmdpM2Ap4FPzazCJl0A\nM5sMnAnsDZwjaaeCMkmVYl5vmYHwMk7SZZLONrNxwOaSJgHPAQ8CjwAHAacBexbUfF128BpvBhW0\nu5nZbdHPwLHAKuA6M3sk2qbCtfFKuhPYHpgP/Bu4hJBITNJzZva9mf0RbXsJ0FbS+Wa2NmNBp5mk\n6sBehMpTP0It93ZgIjDDzD6NtltJGHd6WYZCdYXwm2sZIKk9MNXMlsWtvwh4CGhjZt9I2iz+Jkmu\nkzSAMPD1J0A7QmLpSqi93QyMAfqZ2feSrgTuBdqb2fgMhZwxkrYHxgH3mdkdRWxzHnApcJiZ/ZrO\n+FzRvMabZpJuB64DRkq6A5hkZnMAzOwRSfWBCZKOMLOhFelutKT/AI0LBvaW1BW4G2hoZp9IMqAP\nsCRqyzwf6FiRkq6k84FngDwzmy7p70AfSWMKarnRdvsD+wFXAZ096WYXT7zpNxmYASwCngfGS/oU\neBRYY2Y3SfoD+ETSgWY2MnOhpo+kusARwEOSqpnZSuAbYCVQI2py+TR6qO9hws3H/aP2zQpB0gnA\n44Q2748ljQDeInxuR0oaZ2ZLo2aII6PtDjaz/2UsaFcob2pIE0mbm9lqSdsBQ4Erga+BM4AehPbM\n9wk/G5dLuhD4xMy+zVTM6SLpJkITwizgXcIX0vtAf+BRM7s3bvsOhDm2fkpvpJklqSnhM1lBSLhH\nEmq0TQhf3D3MbFS07RaELor+EE4W8l4NaSDpaeB0ADP7Ebif0Db5p5ndQkjC7YGTgDmSLgeeqCBJ\ntw6wBXAHUBc4HjgbGA28UpB0Y/ugmtnYipZ0AczsF+AioBLwI/AmMBCoAswGHpNUM9p2lSfd7OWJ\nt4xFN4t2N7NnY1YPBX4Dmkk6kND95wJgV0L7b/+Cu/a5zswWA48R+i7/C6gDdAZ+JbTl1o22qxDt\n3PEk3ShpjKQdoiaY0cAgYBczex64EDiMMNPwboS/Ry7LeVNDGZL0DlDfzPaPWVfFzNZKehzoTnhE\n+Hoz65epODMheiovP+b9ZsCdwDFAT+B3Qm3uNcLj0/MzEmgGSfon4SnGO4DNgI+BJwjjedwNXGZm\nUyU1A3YkdCfraWbfZShklyRPvGVE0jOEmx4tYvqcbke4MXQhsAb4CBhhZhdUpP66ku4i3GT8D7DW\nzExSI0Kb92SgJnAFoRY3hlATvqMi1XqjNv6HgQ+B3sCxwKFAK6Ab4cGSfYGDCvovV6QeMOWdNzWU\nnZcId+Q7A0hqAowAJkTtk0uA8YTaS0UbCnIocBNwbJR0mxJGZHuI8GXVn9AG3pDwkMBrFTChvAd8\nBuxJaFp4j9B97j1CMl5EaJY5v2D85gr4GZVbnnjLiJkNJ9RU/hX1vRwDPG5mN0S121WEZHOwpNoV\nafBzM/sQ6AXcFn02o4Enzeyu6NfBq8CnwA3AL2b2Q+aiTb/ocejZhCR7FuELegRQ3cyuJPxiagXs\nBJwAbJ6hUF0JeVNDGYtung0GBplZt5j1lxF+LnYzs+8zFV8mFfbZFPxcjpoe1pjZwowGmUaS9gG+\njGma6glcRqj1PkW4iba/mc2Q1JjQT3dyRf37U5554k0DSfsRnja63szejmp5dxEe46wwDwAUJvps\nngauBz6IHpyocCQ9T/gifobQ4+Xm6AvobmCOmT2kMGbzboQn0SrUr4Bc44k3TSQdRBjoZQJhGMjD\nzezrzEaVHaLP5nHCXfn/VqTBbgpI2pfQdvsisC0g4ANCO3cVM7s02u4jwn2B3WN7hbjyxRNvGkVj\n775KGLB6YqbjySaSDiP8CjgkfvCgiiKq/T8JXE4Y2LwmoZ27LnCEmX0UbdfEzGZlLFBXap5406zg\n0eFMx5GNYsZoqLCiL+eHgZvM7B1J7YAdCI9Qr/KeC7nBE69zWUZSJ0K797Vm9p8Mh+PKgI9O5lyW\nMbNhks4Fnoie6KuQ7d65zGu8zmUpb/fOXZ54ncti3u6dmzzxOudcmvkjw845l2aeeJ1zLs088Trn\nXJp54nXOuTTzxOucc2nmideVmKTJkvrEvP9J0pUZiKODJJPUogzP0UfS5BQcJyOfkcsunnhziKTn\nowRkktZJ+lHS/ZKqpymEjoRRxhKS1EPS8jKOJ/Z8wyQ9mq7zOVccf2Q493xMmERzM+AAwviu1YG/\nF7axpM3MbF0qTlwRJ6R0riS8xpt71pjZXDP7xcxeBV4BukIYfCWqDR8l6UtJawlznCHpWEnjJK2W\nNEPSHZKqFBxUUgNJAyStkjRT0tnxJ47/GS2plqQnJM2JjvutpFOiQWCeA6rH1ND7RPtUkXSPpFmS\nVkj6StIRcefpIum76JifEabAKRVJd0v6Prq+nyTdK+kvU+pIOlfSz9F2/SXViyvvKembKLapki6T\n5P/O3Ea8xpv7VhFqv7HuIczi+wOwLEpsrwCXEOb2akYYF7YqUJBInweaE6afWQk8CLQo6qTRHHLv\nEyZk7AlMBXYmzA82GriUMJ379tEuBc0Oz0XrTgNmAUcBAyV1NLOJ0cSY/Qmjdz1GmJHhX8l/HEVa\nAZwN/ArsQrj+NcCNMdu0AM4AjgeqAX2BfsBx0TX3Am4FLgLGAW2iONcB3szhNjAzX3JkISTHQTHv\n9yRMkf5G9L4TYMCJcfuNAG6MW9eVkAxFqFEasF9MeXPgD6BPzLqfgCuj14cDfwKtioi1B7A8bt32\n0T7N4tb3J0wUCiFZTyV63D1ad0MUX4tiPpthwKOb8FmeD/wQ875PdL3NYtbtH513x+j9z0D3uONc\nCnxT2GfkS8VdvMabe7pEN60qE2q6Awg1sFhj4963B/aU9M+YdXnAFoRpZloREuKXBYVmNlPS7GLi\n2J0wV9i3mxD7HoRE/03cpMtVCbMOE8XyuZnFDjIyZhPOUShJJxGS5A5ADaBStMT61cx+jnn/BdGX\ni6QlQFPgKUlPxGxTmXBNzq3niTf3jCBMK78OmG2F3zhbEfc+D7gFeKuQbedTssRRkn3yCDXIjoT4\nY60qxXGLJWlv4HXCZ3AZsITQfHD/JhymoB23YLp654rkiTf3rLRNn4H2a6BlUftJ+paQWDoSJRVJ\nzYBGCY7ZUFKrImq9a/lrjXI8IbFuY2ZDizjuN8CJkhRT6927mDiSsR+hNntbwQpJzQvZrrGkpmb2\nS/R+T8Ln8q2Z/SbpV2B7M3uxlPG4HOeJ10G4ITRI0kzgTSCfcGNoTzO72sy+lzSE8DO6N6H2+S82\n1EIL8wnhp/h/JV1GaJfdAahuZv0JbZ2bSzqckHBXmtlUSa8Az0u6gpC86xLapn80s7cJN72uAP4t\n6XFgV0ItMxn1ojnMYs2LYmss6XRCs8URwKmF7L8KeEHS5YRmmCeB98xsWlTeB3gkanYYTGjq2QNo\nbGZ3JRmjqwC8m4vDzD4gTDl/MKEd90vgGsLNogI9gBmEttaBhNmSfyrmmH8CRwKjgJeBb4GHgCpR\n+WhC4nqN0JxxdbRrT0LPhnuB74BBwIHAzGi/n4ETgC7ARELTwDVJXuophCQfu1xuZgOB+4B/A5MI\nNwZvKmT/nwhNEgOjz+HHKN6Ca36G0DOiexTbZ4RmnxlJxucqCB8I3Tnn0sxrvM45l2aeeJ1zLs08\n8TrnXJp54nXOuTTzxOucc2nmidc559LME69zzqWZJ17nnEuz/wdhQqODcYyTEAAAAABJRU5ErkJg\ngg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2a3a4806668>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Confusion Matrix\n",
    "conf_mx = confusion_matrix(y_test, y_pred)\n",
    "plot_conf_matrix(conf_mx, train.author.unique(), title=\"MultiNB Pipe Confusion Matrix\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 263,
   "metadata": {},
   "outputs": [],
   "source": [
    "parameters = {\n",
    "    'vect__max_df': (0.5, 0.75, 1.0),\n",
    "    'vect__max_features': (None, 5000, 10000, 50000),\n",
    "    'vect__ngram_range': ((1, 1), (1, 2)),  # unigrams or bigrams\n",
    "    'tfidf__use_idf': (True, False),\n",
    "    'tfidf__norm': ('l1', 'l2'),\n",
    "    'clf__alpha': (0.00001, 0.000001),\n",
    "    'clf__penalty': ('l2', 'elasticnet'),\n",
    "    'clf__n_iter': (10, 50, 80),\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 264,
   "metadata": {},
   "outputs": [],
   "source": [
    "sgd_clf = Pipeline([\n",
    "    ('vect', CountVectorizer()),\n",
    "    ('tfidf', TfidfTransformer()),\n",
    "    ('clf', SGDClassifier())\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 265,
   "metadata": {},
   "outputs": [],
   "source": [
    "sgd_clf_gs = GridSearchCV(sgd_clf, parameters, n_jobs=-1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 266,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(14684,) (4895,) (14684,) (4895,)\n"
     ]
    }
   ],
   "source": [
    "print(X_train.shape, X_test.shape, y_train.shape, y_test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 267,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\anoop\\Anaconda3\\lib\\site-packages\\sklearn\\linear_model\\stochastic_gradient.py:117: DeprecationWarning: n_iter parameter is deprecated in 0.19 and will be removed in 0.21. Use max_iter and tol instead.\n",
      "  DeprecationWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=None, error_score='raise',\n",
       "       estimator=Pipeline(memory=None,\n",
       "     steps=[('vect', CountVectorizer(analyzer='word', binary=False, decode_error='strict',\n",
       "        dtype=<class 'numpy.int64'>, encoding='utf-8', input='content',\n",
       "        lowercase=True, max_df=1.0, max_features=None, min_df=1,\n",
       "        ngram_range=(1, 1), preprocessor=None, stop_words=None,\n",
       "        strip...='l2', power_t=0.5, random_state=None,\n",
       "       shuffle=True, tol=None, verbose=0, warm_start=False))]),\n",
       "       fit_params=None, iid=True, n_jobs=-1,\n",
       "       param_grid={'vect__max_df': (0.5, 0.75, 1.0), 'vect__max_features': (None, 5000, 10000, 50000), 'vect__ngram_range': ((1, 1), (1, 2)), 'tfidf__use_idf': (True, False), 'tfidf__norm': ('l1', 'l2'), 'clf__alpha': (1e-05, 1e-06), 'clf__penalty': ('l2', 'elasticnet'), 'clf__n_iter': (10, 50, 80)},\n",
       "       pre_dispatch='2*n_jobs', refit=True, return_train_score='warn',\n",
       "       scoring=None, verbose=0)"
      ]
     },
     "execution_count": 267,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sgd_clf_gs.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 268,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.81742032143830023"
      ]
     },
     "execution_count": 268,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sgd_clf_gs.best_score_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 269,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "clf__alpha: 1e-05\n",
      "clf__n_iter: 80\n",
      "clf__penalty: 'l2'\n",
      "tfidf__norm: 'l2'\n",
      "tfidf__use_idf: True\n",
      "vect__max_df: 1.0\n",
      "vect__max_features: None\n",
      "vect__ngram_range: (1, 2)\n"
     ]
    }
   ],
   "source": [
    "for param_name in sorted(parameters.keys()):\n",
    "    print(\"%s: %r\" % (param_name, sgd_clf_gs.best_params_[param_name]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 270,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = sgd_clf_gs.best_estimator_.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 271,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy score: 0.845965270684\n"
     ]
    }
   ],
   "source": [
    "print(\"Accuracy score:\", accuracy_score(y_test, y_pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 272,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV4AAAEqCAYAAABdmDjNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XeYVEXWx/Hvb4acFASUjIIKooKK\nWcyuWVlQ0TVhwrDmtK6i4Jpzzr6KaXcRA4qYMQBmUFFRyaDEBYQhDJnz/lF3oGlnppuZnu6envPZ\n5z4zXXXv7brtcrrm3LpVMjOcc86lT16mG+Ccc1WNB17nnEszD7zOOZdmHnidcy7NPPA651yaeeB1\nzrk088BbRUkaK2n/TLdjY0m6WdI8SbPLcY7WkpZIyk9l29JN0rWSns50O9zG88CbBpL2kfS5pAJJ\nf0j6TNKuMfXNJD0laWYUECZLGiCpQ1TfVpJFdUskzZH0lqRDEryvSVoaHTND0r1FwcbMOpnZJxV0\nvbtJelvSwuh6v5Z0RgrO2wq4AtjOzLYo63nM7Dczq2dma8rbpnjRZz5HUrWYsmqS/icpqUHzkvaX\nND3RfmZ2q5mdXZ72uszwwFvBJDUA3gIeAhoBLYAbgRVR/WbA50AdoBtQH9gZ+BSID6ybmlk9oDPw\nAfC6pN4JmtA5OuYg4G/AOeW/qpJJ2hP4iND+9sBmwPnA4Sk4fRtgvpn9LwXnqkgL2fB6jwAWpPIN\nYgO7q4TMzLcK3ICuwMJS6m8GxgB5pezTFjCgWlz5lcCcko6Njmkf83oQ8HD0+1Tg4Oj3/sArwEBg\nMfAtIWAXHdcceBWYC0wBLi6lrSOBRxJ8JucAE4E/gDeB5nFtPg+YQAhWjwACDgaWAWuBJcAAYH9g\nety5Y69rN2AUsCj6nO4t7vOMru/NqD0TgXNiztcfeBl4PvpsxgJdS7k2A/oCg2LKXgGuC//c1pWd\nAfwSnXMycG5UXjfuOpdE7Sv6b/RidD1nR2UvRsf1is7TIHp9ODAbaJLpfwO+FfP/k0w3INc3oAEw\nH3gu+sfQMK7+S6B/gnNsEChiyreKyjuWcNy6wAtsF/1DPCt6HRug+gOrgOOA6oSAPiX6PQ8YDdwA\n1IjeczJwaDHvVwdYAxxQyrUcCMwj9OprEv4SGB7X5reATYHWhGB/WFS3PzGBNv51Mdf1BXBq9Hs9\nYI/iPk9C7/xRoBbQJXrPg2I+m+WEXms+cBvwZSnXZ8D2hEC/abTNicosZr8jgXaEL5X9gEJg51Ku\nq+i/Uffov0ltYgJvtM9LhC+kzYCZwFGZ/v+/b8VvnmqoYGa2CNiH8A/yKWCupDclbR7t0pgQEAGQ\ndEyUG10s6f0Ep58Z/WxUyj7fSloADAGeBp4tYb/RZvaKma0C7iUEoT2AXQm9pn+Z2Uozmxxdx4nF\nnKMhISjMKqU9JwPPmNm3ZrYC+Cewp6S2MfvcbmYLzew34GNCMCyLVUB7SY3NbImZfRm/Q5Q33gf4\nh5ktN7PvCZ/TqTG7jTSzty3khF8gpHpKs5zwefcifE5vRmXrmNlQM5tkwafA+4RUU2m+MLPBZrbW\nzJYVU/93whfbJ8AQM3srwflchnjgTQMz+8XMeptZS0LPpzlwf1Q9H2gWs++bZrYpcBmhh1maFtHP\nP0rZZ2cza2hm7cysr5mtLWG/32PasBaYHrWzDdA8+jJYKGkhcC2weTHnWED4E7lZMXVFmgPTYt5r\nCeEzaBGzT+yIhUJCb7UszgK2AX6V9I2ko0pozx9mtjimbFqC9tRKIsf6PHBatD0fXynpcElfRjcf\nFxJ61I0TnPP30irNbCEhnbQ9cE+Cc7kM8sCbZmb2K+HPwe2jomFAd0ll+W/xV+B/wLgUNK1V0S9R\nW1oSetS/A1PMbNOYrb6ZHRF/AjMrJPx537OU95lJCOZF71WX8KfxjDK0eSkhvVF0rnygSUx7JpjZ\nSUBT4A7glej94tvTSFL9mLLWZWxPrBGEL6DNCXnvdSTVJOTM7wY2j75o3yakHSD8dVScUkdFSOoC\nnAn8B3iwzC13Fc4DbwWT1EHSFZJaRq9bAScRcrsQ/qxvCLwgqZ2C+pTy57WkzSVdCPQD/llKL3Zj\n7CKpR9STu5Qw6uJL4GtgkaR/SKotKV/S9rHD4eJcDfSWdFU0YgNJnSX9N6r/N3CGpC5RALoV+MrM\nppahzeMJvc8jJVUn3NSqWVQp6RRJTaLPZ2FUvMEQMjP7nTCq5DZJtSTtSOgpv1SG9sSe14CjgWOi\n32PViNo5F1gt6XDgLzH1c4DNJG2S7PtJqkW48XYt4cZdC0kXlOMSXAXywFvxFgO7A19JWkoIZj8R\nxqNiZvMIudTlhJ7RYuB7wrCy8+POtTA6x4+EP02PN7NnUtTONwg5yQWE/GYPM1sV5TWPJnwRTCHc\nGHsaKDYomNnnhDzjgcBkSX8ATxJ6dJjZMOB6Qo9vFuEGU3H54oTMrAC4IGrPDEIPOHb862HAWElL\ngAeAE81s+Z9OFL4I2xJ6v68D/czsg7K0Ka59Y81sbDHli4GLCaMlFhCG+b0ZU/8rodc6OUrvNE/i\n7W4j3JB7LMqdnwLcLGnr8l6HSz39+cvYVTWS+hNGP5yS6bY4VxV4j9c559LMA69zzqWZpxqccy7N\nvMfrnHNpVmUn2lC12qYa9RPvWAV17tA6001wldBvv01l/rx5Srxn8vIbtDFbXdxDehuyZXPfM7PD\nUvneFanqBt4a9am57QmZbkZW+vizBzLdhKzlmbmSHbjP7ik/p61eltS/0+XfP5Loqb+sUmUDr3Ou\nMhCU6aHO7OaB1zmXvQTkVeqFQorlgdc5l92U0rRxVvDA65zLYp5qcM659PMer3POpZHwHq9zzqWX\n/Oaac86lXQ6mGnKvD++cyyHRzbVEWzJnki6UNErSCkkD4urqSHpU0jxJBZKGx9RJ0h2S5kfbndL6\nb4NoUv/RkgqjnwnXCPTA65zLXiL0eBNtyZkJ3AwUt3jAk4RFYztGPy+LqetDWN25M7AjcBRwLoCk\nGoRFBF4krCTzHPBGVF4iD7zOueyWoh6vmb1mZoMJi6uuP720LXAM0MfM5prZGjMbHbPL6cA9Zjbd\nzGYQFhLtHdXtT0jZ3m9mK8zsQcLXxYGltcUDr3Muiwny8xNv0DhKIxRtfTbiTXYnrCx9Y5Rq+FFS\n7IKtnYAxMa/HRGVFdT/Erav3Q0x9sfzmmnMueyU/nGyemXUt47u0JKz6/SrQHNgTGCrpZzP7BagH\nFMTsXwDUi/K88XVF9aVOfeg9Xudcdktdjrcky4BVwM1mttLMPgU+Zv3Kz0uABjH7NwCWRL3c+Lqi\n+sWlvaEHXudcFkvdqIZS/JCgfizhxlqRzlFZUd2OsaMcCDfg/rS6dCwPvM657JaiHq+kapJqAflA\nvqRakqoBw4HfgH9G++xNuGn2XnTo88DlklpIag5cAQyI6j4B1gAXS6op6cKo/KPS2uKB1zmXvRQ9\nuZZoS05fQlrhGuCU6Pe+ZrYKOBY4gpCffQo4zcx+jY57AhgC/Aj8BAyNyjCzlYShZqcBC4Ezge5R\neYn85ppzLrulaK4GM+sP9C+hbizhplpxdQZcHW3F1X8H7LIxbfHA65zLbjn4yLAHXudcFvP5eJ1z\nLv28x+ucc2kkQV7uhancuyLnXG7xHq9zzqWZ53idcy7NvMfrnHNpJB/V4Jxzaae83Au8uXdFWeLx\nficzbdhtjBp07Qbl55+4H2Nev57Rr1zHLZccC0D1avk80f8Uvnn5Wr4aeA3ddtl63f4nHLYL37x8\nLV8P/CdvPHwBm21aN63XUdEuPPdstm7TjD27rp+D5JYbb2Dv3Xai2+670OPow5g1cyYAD953N912\n34Vuu+/Cnl07s1m9Giz4449MNb3CXXje2WzTphl7de38p7qH7r+HRnWrMX/ePAAWFRRw0nHH0m33\nndmz64689PyANLe2YoQFKJRwq2w88FaQF4Z8ybF/f2SDsn27bs1R++/Arifcxi7H3cL9zw8D4Mwe\newOw6wm3ctR5D3P75X9FEvn5edx11XEc1ucBdut1Gz9NmMF5vfZL+7VUpJNOPY1XBg/doOyiy67k\ns6+/Y8RXozn08CO587abAbj4sisZ8dVoRnw1mhtuvJm9u+1Lw0aNMtHstPjbKacxKO6zAZg+/Xc+\n+ehDWrZqva7s6ScfZdsOHRnx1bcMeWcY1197FStXljpdQOWgJLdKxgNvBfns20n8UVC4QVmf47tx\n97MfsHLVagDmLlgCQIettuDjr8etKytYvIxdtmu9buKlurXD8k3169Vm1tz4OZcrt733+XPwbNBg\n/fSmS5cuLbZH8+qggfQ8/sQKb18m7VXMZwNw3T+u4Mabb9/gcxFiyZIlmBlLly6hYcNGVKuWC5nE\nxL1d7/G6UrVv05S9d2rH8Oev5P2nL2GX7UKP5cfxMzh6/x3Iz8+jTfPN2Gm7VrTcoiGrV6/lklsH\n8s3L1zL5/VvouNUWDBj8eYavIj1u6teXTlu3ZdDA/3Dt9f03qCssLGTYB+9xTPcemWlcBr0zdAjN\nmrVg+x03TD+cfd7fGT/uF7Zr14p9duvCbXfdS16O5EY98FYQSVMlLZO0JGZ7OKZ+f0km6eq449pG\n5UXHTJV0TfqvIDnV8vNo2KAO+552N9feN5gX7zwTgOfe+IIZcxby2UtXc9dVPflyzBRWr1lDtWp5\nnHNcN/Y46Q62+st1/DR+Bled+ZcE75Ibrr/xZsZOmMrxvU7iqcc3TNm8+/Zb7L7HXjmdZihOYWEh\n99x565++iAA++vB9tt+hMz9P+p1PvxjN1ZdfwqJFi9LfyAqQl5eXcKtssqnFR5tZvZjtwpi604E/\nop/F2dTM6gEnATdIOqyiG1sWM+YsZPCwsGbeqLHTWLvWaNywHmvWrOXqe15jjxNv54TLnmTT+rWZ\n+NtcOm/TEoAp08MNlFc++JY9Om+VsfZnwnG9TuLNN17foOy1QQPpeUJupxmKM3XyJH6bOpVue+xM\n547tmDljOvvvvStzZs/m3y8M4Ohjw72Brdq1p02btkwY/2vik2Y7z/FmhqQ6wHHA34GtJZW4oJ2Z\nfUFYcmP7NDVvowz55Af2320bANq3bkqN6tWYt2AJtWtVp06tkMc9cPcOrF6zll8nz2bm3AI6bLUF\njRvWA+CgPTowbsrsjLU/XSZNnLDu93eHDmGbbbZd97qgoIDPRg7niKOOyUTTMmq77Xdg/LRZjPll\nEmN+mUTzFi355LNv2HyLLWjZqjWffhIWPfjfnDlMnDCetm0r/5e0cjTHWxmy7z0JC8oNAk4mzPQ+\nKn6naM2jvQjLKn9X3ImiJZ/Dss/V61VMayPP3dabbrtsTeNN6zHx3Zu46fG3eW7wFzzR/2RGDbqW\nlavWcPYNLwDQpGF9hjz6d9auNWbOXchZfZ8DYNbcAm598h0+ePpSVq1ew2+z/qBPvxcrtN3pdtbp\nJ/PZ8E+ZP38endq34Zq+/fjgvXeYMGE8eXl5tGrVmnsffHTd/kPfHMwBBx1C3bq5NayuOGeffjKf\njYg+m63DZ3Pq6WcWu++V11zH3/ucyd67dsHM6HfTbWzWuHGaW1wxUhVYo2V5egM7AP8xs97F7NOP\nMFn6IWb2YVRWE3iM0AEsBO40s3tjjjkIeARoDXwF9DazaaW2ZcPl4DND0lSgMbA6pvgqM3tK0ofA\nT2Z2qaSTgAeB5ma2SlJbYAphuQ4DZgOPmdmDid4zr05Tq7ntCam9kBwx6/MHMt2ErJUF/1yy1oH7\n7M53345Kafez2mZbWYMjbk6434IXTx6daHl3ST2AtcChQO34wCupHTAY2Iyw9E9R4L0N2Ac4BtiC\nsAJxbzN7V1JjYBJwNmF5oJuAbma2R2ltyaZUQ3cz2zRme0pSK+AA4KVonzeAWsCRccc2NrOGZtYx\nmaDrnKskBMpTwi0ZZvaamQ0G5pewy8PAP4D4AdCnATeZ2QIz+4WwJlvvqK4HMNbMBpnZckJvubOk\nDqW1JZsCb3FOJbRxiKTZwGRC4D0to61yzqVNOnK8ko4HVprZ23HlDYHmwJiY4jGElCbRz3V1ZraU\n0APuRCmyPcd7GnAj8HhM2W7AIEmbZaZJzrl0Kbq5loTGkmLv/TxpZk8m9R5SPeBWoLixmkU3g2Kf\nXCoA6sfUz407Jra+WNkUeIdIWhPz+hugLfCImcVe2JuSJhKGjr2VxvY55zIgycA7L1GOtxQ3Ai+Y\n2ZRi6pZEPxsAy2N+XxxT3yDumNj6YmVF4DWzthu5f2w3vvKNJXHOJa/i/4UfBLSUdEH0ugnwsqQ7\nzOwOSbOAzsAHUX1nwrBVop/rni+QVBdoF1NfrKwIvM45VyyRsifTJFUjxLx8IF9SLcJIqoOA6jG7\nfgNcDrwTvX4e6BulMjYHzgHOiOpeB+6S1BMYCtwA/GBmpT69ku0315xzVVwKb671BZYB1wCnRL/3\nNbP5Zja7aAPWAAvMrCjN0I9ww2wa8Clwl5m9CxClQXsCtwALgN2BhI9Veo/XOZe1NuLmWkJm1p8w\n3CvRfm3jXq8Azoy24vb/ECh1+Fg8D7zOueyWg3dxPPA657KXUvfIcDbxwOucy2qVcdrHRDzwOuey\nW+51eD3wOueym6canHMujSrrfLuJeOB1zmU1D7zOOZdmyU77WJl44HXOZTXv8TrnXDr5OF7nnEsv\nATkYdz3wOueymY9qcM65tMvzm2vOOZdG8lSDc86llfAer3POpZ33eJ1zLs1y8eZa7s235pzLGVJI\nNSTakjuXLpQ0StIKSQNiyveQ9IGkPyTNlTRIUrOYekm6Q9L8aLtTMd8GkrpIGi2pMPrZJVFbPPA6\n57JY4vXWNqJHPBO4GXgmrrwh8CTQFmhDWJr92Zj6PkB3wurCOwJHAecCSKoBvAG8GJ3nOeCNqLxE\nHnidc1lNSrwlw8xeM7PBwPy48nfMbJCZLTKzQuBhYO+YXU4H7jGz6WY2A7gH6B3V7U9I2d5vZivM\n7EHCPcEDS2uLB17nXFZLssfbOEojFG19yvGW+wJjY153AsbEvB4TlRXV/WBmFlP/Q0x9sfzmmnMu\neyXfo51nZl3L/XbSjsANwLExxfWAgpjXBUC9KM8bX1dUX7+09/HA65zLWukcxyupPfAOcImZjYip\nWgI0iHndAFhiZiYpvq6ofnFp7+WpBudcVkvhzbXS3qMN8CFwk5m9EFc9lnBjrUhn1qcixgI7asNG\n7MiGqYo/8cDrnMtqqbq5JqmapFpAPpAvqVZU1gL4CHjEzB4v5tDngcsltZDUHLgCGBDVfQKsAS6W\nVFPShVH5R6W1pcqmGnbs0Iphw+/PdDOy0mEPjMx0E7LWJ1ful+kmZK0KyQikdj7evkC/mNenADcC\nBmwF9JO0rt7M6kW/PhHV/xi9fjoqw8xWSuoeld0O/AJ0N7OVpTWkxMArqU6yVxMNwXDOuZRK5Xy8\nZtYf6F9C9Y2lHGfA1dFWXP13wC4b05bSerxLCN8EycjfmDd1zrnkJP9kWmVSWuA9PG2tcM65EuTi\nXA0lBl4zey+dDXHOuT/J0fl4kx7VIGmzaJKJ+yRtFpXtKqlVxTXPOVeVhRxvxQ8nS7ekRjVI6gwM\nA+YC7YCHCM87H0OYWOLUCmqfc66Kq4yBNZFke7z3AE+bWUdgRUz5O0C3lLfKOeciqZoWMpskO453\nV6Jp0OLMALZIXXOccy5GjuZ4kw28K/jz88gA2wDzUtcc55xbTzm6vHuyqYYhQF9JRYHaosfsbgde\nr5CWOeccqXtkOJskG3ivBFoCs4HahOeQJwMrgWsrpmnOOQd5UsKtskkq1WBmCyTtCRwG7EwI2N8C\nb5vZ2gpsn3OuCitacy3XJD1JThRg344255xLixyMuxv1AMVhkt6XND3aPpDkjxU75ypULj5AkVTg\nlXQR8CYwB7g12mYDgyVdXHHNc85Vdbl4cy3ZVMM1wGVm9khM2aOSviDMcflgylvmnKvyRBhSlmuS\nTTVsAhQ3ac77FD++1znnyk8iPy/xVtlszDjeY4spPxa/2eacq0ApXPrnwmjp9xWSBsTVHSTpV0mF\nkj6O1mArqqsp6RlJiyTNlnR5sseWpLQVKC6Iefkj4QGKbsCXUdkewH7AnQmv2DnnykCQynG6M4Gb\ngUMJzyOE95AaA68BZxM6mTcBAwkxDsKqFVsDbQhTJHws6WczezeJY4tVWo73+rjXy4Hdoy227CLg\nttLexDnnyiqFS/+8Fs6nroQHwor0AMaa2aCovj8wT1IHM/sVOA04w8wWAAskPQX0Bt5N4thilTYR\nerMyX6FzzqVIksPFGksaFfP6STN7Msm36ASMKXphZkslTQI6SZoDNI+tj37vnuhYYOMDr3POZZpE\nsjfP5plZ1zK+TT3CXOOxCoD6UV3R6/i6RMeWKOnAK6ktoVvdGqgRW2dmFxRziHPOlVsaxiws4c+j\nsxoAi6O6otfL4+oSHVuiZFegOITwAMWvrO9ab0VYXfjrZM7hnHNlkYYn08YCp8e8X13CSjtjo3lq\nZgGdgQ+iXTpHx5R6bGlvmOxwsluB281sJ8LcvL0IPd9PCQHZOedSLoxqSLwldS6pmqRahA5jvqRa\n0VS3rwPbS+oZ1d8A/BBzc+x5wqiuhpI6AOcAA6K6RMcWK9nA2wF4Mfp9NVDbzJZGb3JVkudwzrmN\nk8Q8DRvRI+4LLCM8iXtK9HtfM5sL9ARuARYQRm6dGHNcP2ASMI3Q2bzLzN4FSOLYYiWb413K+rzu\nLNZ3pQ3YLMlzOOfcRkvVtJBm1p8wJre4ug8JHczi6lYAZ0bbRh1bkmQD79fAXsAvhLFrd0nqSIj0\nnuN1zlWIolRDrkk28F7F+mEV/YCGwFnAeMIDFK4UF59/Nu+/+zaNmzRl5NffA3DW6X9j0oRxABQU\nFLDJJpvwyeejAbj/7jt46YVnycvL57a77uPAg/+SsbZXhOuO2Ia9223GgsJVnPx/Yejlgds25ux9\n2tK2cR3OfO5bfp0dbiY326Qm/zl7V377YxkAP81cxJ3vTaBOjXweP7nLunM2rV+Td8fO4f5hk9J/\nQWny4P33MeDZp5FEp+134Mmnn6VWrVoAXHbJRbzw3LPMW7gkwVkqn8o47WMiya5AMS7m98XAGQCS\nqgOtKqZpuePEk0/nrHMv4O991v+l8n/P/Xvd79f/8yoabLIJAON+/ZnXXx3IyK/HMHvWTHoecxhf\nffcz+fn5aW93RRn64xxeGT2TG45a/9fZ5HmFXPP6WK45bJs/7T9j4XJOe3b0BmWFK9dsUDag9858\nMj53112dMWMGjz7yIN/98DO1a9fm5JNOYNDA/3Lq6b0ZPWoUBQsXZrqJFSb3wu5GTIRegu2ACalo\nSC7ba59uNGzYqNg6M+ON11+hx3G9AHjnrSH8tWcvatasSZu2W7LlVu34dlRuZXO+/72ARctXbVA2\ndX7hul7txmrVsDYN61Tn+98LEu9cia1evZply5aFn4WFNGvenDVr1nDtNVdxy+25OWWKlJtrrpU3\n8Lpy+uKzkTRp2pR27bcGYNasGTRvuf4x8ubNWzBr1sxMNS8rNN+kFs+dsTOP/q0znVtu8qf6Q7Zr\nyoe/xD88lFtatGjBpZddyTZbtWbLVs1o0GATDj7kLzz2yMMcedQxNGuWu0/45+Up4VbZZEXglTRV\n0sFxZb0ljYypXyZpiaQ5kp6VVC+q+0TS2Zlodyq89sp/6XHc+tEnZvanfXIxx5WseUtWcuyjX3L6\ns9/ywLBJ/OuYDtSpsWHa5ZCOTXj/5/9lqIXpsWDBAt4a8ga/TJjC5N9msrRwKS+98DyvvTqICy7M\n7dssubgCRVYE3iQdbWb1CKsc70oYk1eprV69mqFvDuavPY9fV9a8eUtmTp++7vXMmTPYYovc7c0k\nsmqNsWj5agDGzVnCjIXLad1o3Yx+tG9al/w8MW5O7t1UivXRsA9p23ZLmjRpQvXq1enevQc3/asf\nkydNpFOH9mzbvi2FhYV06tA+001NKZE4zZBzqQZJ25W2EcbzppWZzQDeAbZP93un2qcfD6P9NtvS\nvMX61MJhRx7F668OZMWKFUybOoXJkyayc9fdMtjKzNq0dvV1w4mab1KLlg1rM3Ph8nX1f+nYNOd7\nuwCtWrXm66+/pLCwEDPj44+GcfEllzN1+mzGTZzKuIlTqVOnDmN/nZjppqZWEr3dShh3E45q+Inw\nkERJlKA+5SS1Ao4gTD68scf2AfoAtGzVOsUtK9k5Z5zCZyM+5Y/589hh27b849obOOX0M3n9lYH0\nOL7XBvt26NiJY3scz9677kh+fjXuuOfBnBrRAPCvYzqyc+tN2LR2dd68YA+eGjmVRctXccXBW7Np\nnerce/wOjJ+zhEtf/pGdWm/COfu0ZY0Za9cad743YV0PGOCgjk24/OUfM3g16bHb7rvz1x7Hsedu\nO1OtWjU6d96Js87pk+lmpUUuptpUXE5xXaW0bTIniR1uVqZGSFOBxoTHkYvUAL41s33i6guAocAV\nZrZM0ifAi2b29Ma8Z5edd7Fhw78qT7Nz1pEPf5bpJmStT67cL9NNyFp7796V0aNHpTRKbt5+e+t1\n9ysJ93vorx1Hl2NayLQrtcdb3oC6kbpHj94B4eYaYTmNYuudc1VDJRy0kJBPhO6cy2oeeLNX0XRv\nRdaY2aoS93bOVQrh5lnuRd7KNJysNI8Rpngr2p7NbHOcc6mSqvl4s0lW9HjNrG0xZQOIJhsurj5m\nv/0rplXOuUwTSa+5VqlsVI9XUj1JnaPJcZxzrsLlJbFVNkm1WVJdSc8Di4DRRDOSSXpY0nUV2D7n\nXBWXqgcoJLWV9LakBZJmR/GrWlTXRdJoSYXRzy4xx0nSHZLmR9udKmfiOdkvi9sIM6zvxfqVNgHe\nB44v9gjnnCsnJfG48EY8Mvwo8D+gGdAF2A+4QFIN4A3C8mYNgeeAN6JyCA9ddScscrkjcBRwbnmu\nK9nAeyxwkZl9yYZPqv1MWG3YOecqRAofGd4SeNnMlpvZbMJqOp2A/Qn3u+43sxVm9iAhvXxgdNzp\nwD1mNj2asuAeoHd5rinZwNuE8E0Rr2553tw550ojoFqeEm5JegA4UVIdSS2Aw1kffH+wDR/j/SEq\nJ/o5JqZuTExdmSQbeEcT5keeG4NAAAAcRElEQVQoUtTAM4EvytMA55wrTZI93saSRsVsxU1k8Skh\nYC4CpgOjgMGEZc3iZ9EvAOpHv8fXFwD1ypPnTXY42XXA29Ga8tWAv0sq6qL7w+vOuYqR/DjdeaXN\n1SApD3gPeIJwr6oe8AxwB2Hl9AZxhzQAFke/L4mrbwAssdImukkgqR6vmQ0nBNimwAygB2HJ973N\nLLfWpXHOZRUl8b8kNCKMxno4yuPOJzxodQQwFtgxrge7Y1RO9LNzTF3nmLoySfoBCjMbDfRKuKNz\nzqVIqpZ3N7N5kqYA50u6m9DjPZ2Qr/0EWANcLOlx4JzosI+in88Dl0t6m5BmvQJ4qDztSXYcb53S\ntvI0wDnnSpOfp4RbknoAhwFzgYmEaWYvM7OVhOFipwELCfeuukflENITQ4AfCXOUD43KyizZHu8S\nSp/wPLdm6nbOZYVU9XgBzOx7wn2p4uq+A3Ypoc6Aq6MtJZINvIfHva4O7ESYL/f6VDXGOec2UEmX\n9kkkqcBrZu8VU/yWpPHAKYQciHPOpVxlXMwykfLOLzGK9U93OOdcShWlGnxayEj0HPPfCcPLnHOu\nAoj8HOzxJhV4Jc1lw5trAjYFVhLuBDrnXMqJKpzjBfrGvV5LGJLxuZkVN4eDc86VXyVNJSSSMPBG\n81WuAt6OZvRxzrm0qZI318xsNfAwULPim+Occ+sVpRpSNC1k1kg21fA14fnkaRXYFuec+5NcXHMt\n2cD7MHCPpOaEKSKXxlaa2c+pbphzzonKuaZaIskG3pejn49GP4tGOCj63R8Zds6lnsLyP7km2cDb\nsUJb4ZxzJci9sJsg8Ep6BrjEzMalqT3OObdOeHIt90JvovTJ6UDtdDTEOeeKUxUfGa6El+Scyx2q\nsjneMq8r5Jxz5VGVRzXMTvSNY2Y+qsE5VyGqao+3D2E5DOecS7tUhl1JJwL9gNbAbKC3mY2QdBDw\nSFT+VVQ+LTqmJvAYcBxQCNxpZveWpx3JBN4huTgRjhD5+bn3TZoKn1y5X6abkLU6/ePtTDcha02f\nUZDyc0qkbFpISYcQlnPvRXgat1lU3hh4jbCizhDgJmAgsEd0aH9ga6ANsAXwsaSfzezdsrYlUeD1\n/K5zLqNSmGq4EfiXmX0ZvZ4Rnb8PMNbMBkWv+wPzJHUws18JU9+eYWYLgAWSngJ6A2UOvIny1t4l\ndM5llJLYgMaSRsVsfTY4h5QPdAWaSJooabqkhyXVBjoRlnkHwMyWApOATpIaAs1j66PfO5Xnmkrt\n8ZpZLt5QdM5VIkl2eOeZWddS6jcnLNJ7HNCNMNXtG4S5xusR5hePVQDUj+qKXsfXlZkHVudc1grD\nyZRwS8Ky6OdDZjbLzOYB9wJHAEuABnH7NwAWR3XE1RfVlZkHXudcFhN5SrwlEuVnp1P8fauxhGlv\nwztKdYF2hLzvAmBWbH30+9hyXJQHXudcdkvhROjPAhdJahrlbi8F3gJeB7aX1FNSLeAG4IfoxhrA\n80BfSQ0ldQDOAQaU55o88DrnslYKUw0Qhol9A4wHfgG+A24xs7lAT+AWYAGwO3BizHH9CDfbpgGf\nAneVZygZlGN5d+ecq3ApXNrHzFYBF0RbfN2HQIcSjlsBnBltKeGB1zmX1XLwiWEPvM657CVS9+Ra\nNvHA65zLasrB57g88DrnsloOdng98Drnspv3eJ1zLo3CmmuZbkXqeeB1zmWvJJ9Mq2w88Drnslru\nhV0PvM65LJary7t74HXOZbXcC7seeJ1z2S4HI68HXudcVvNUg3POpVnuhV0PvM65bJeDkdcDr3Mu\na4XFLHMv8nrgdc5lrxTOx5tNfAUK51xWS+HSP9H5tLWk5ZJejCn7m6RpkpZKGiypUUxdI0mvR3XT\nJP2tvNfkgdc5l8WU1P820iOEJYDCO0idgCeAUwnLwBcCj8btvzKqOxl4LDqmzDzV4JzLaqlMNUg6\nEVgIfA60j4pPBoaY2fBon+uBXyTVB9YS1mPb3syWACMlvUkI0teUtR0eeNPgwnPP5r13h9K4SVO+\nGDUGgFtuvIG3hw4hT3k0adqER554hmbNm/PgfXcz6L//AWD1mtWM//UXJv42m4aNGpX2Fjlh/Lhx\nnPq3XuteT5kymev7/Yt999ufi/5+HiuWL6datWrc/9Cj7LrbbhlsacW5vdcOHNixKfOXrOTwu0cA\ncM1RHTiwU1NWrV7Lb/MLufq/P7B4+Wqq5YnbTtiBTi03IT9PvD5qBo9/NAmA+rWqcdsJO7BNs/qY\nwTUDf+C7aQszeWllIpIe1NBY0qiY10+a2ZMbnEtqAPwLOAg4K6aqEyEQA2BmkyStBLYhBN41ZjY+\nZv8xwH7JX8WfeaohDU469TReGTx0g7KLLruSz77+jhFfjebQw4/kzttuBuDiy65kxFejGfHVaG64\n8Wb27rZvlQi6ANtsuy1fjf6er0Z/z+dfj6ZOnToc0/2vXPfPq7nu+n58Nfp7ru//L67759WZbmqF\nefWb6Zzx1DcblI0cP4/D7xrBkfeMZMrcpZx/UDsADu/cjBrV8jji7hEce99ITtqzFS0a1gbghu7b\nMXzcXP5yx3COumcEE+csSfu1pIyS2GCemXWN2Z4s5kw3Af9nZr/HldcDCuLKCoD6CerKzANvGuy9\nz5+DZ4MGDdb9vnTpUlTM31OvDhpIz+NP/FN5VfDxR8PYcqt2tGnTBkksWrQIgIKCApo1b57h1lWc\nbyYvYGHhqg3KRo6fx5q1BsD30xayxaa1ohqjdo188vNErer5rFpjLFm+mno1q7HrVo14+avpAKxa\nYyxevjqdl5FSedHUkKVtiUjqAhwM3FdM9RKgQVxZA2Bxgroy81RDBt3Ury///feLNNhkE4a88+EG\ndYWFhQz74D3uuvfBDLUuswYN/C8n9DoJgLvuuZ+jjzyUf/7jStauXcvHwz9PcHTuOm63lgz9fhYA\n74yZzcGdNueLfgdSu3o+t7z5CwXLVtGxeX3+WLqSO0/ckQ7N6/PT9EXcNPhnlq1ck+HWl02KUrz7\nA22B36JOTj0gX9J2wLtA53XvJ20F1ATGE1IN1SRtbWYTol06A2PL05i09XglTZW0UlLjuPLvJZmk\nPSUtkpQfU/dUCWWPR793kvS+pAWSFkoaLemIdF1TeV1/482MnTCV43udxFOPP7JB3btvv8Xue+xV\nZdIMsVauXMnQt96kx3HHA/DkE49x5933MXHK79x5932c3+esBGfITRcc1I41a403vp0JQOfWm7LG\njL1u/Ij9b/2Es/bbklaNalMtL49OLRrw0ufTOObez1i2YjXnHbhVZhtfVsmkGZKLzE8C7YAu0fY4\nMBQ4FHgJOFpSN0l1CXng18xssZktBV4D/iWprqS9gWOBF8pzWelONUwBTip6IWkHoHb0ch6QD+wc\ns383YGZc2b7A8Oj3IcAHhGEeTYGLgUUV0fCKdFyvk3jzjdc3KHtt0EB6nlA10wzvvfsOXXbamc03\n3xyAl154ju5/7QFAz+OOZ9Q3X2eyeRnRo2sLDtiuKZe99P26sqN3bs7wX+eyeq0xf8lKRk9dwA6t\nNmFWwTJmFyxnzG8hNfnOD7Pp1GKTTDW93FIxnMzMCs1sdtFGSCEsN7O5ZjYWOI8QgP9HyN9eEHP4\nBYQ49T/gP8D50TFllu7A+wJwWszr04Hno99XAV8SAiuSmgI1gIFxZdsAw6Oe85bAU2a2Mto+M7OR\nabmScpo0ccK6398dOoRtttl23euCggI+GzmcI446JhNNy7iXB/5nXZoBoFnz5owY/ikAn3z8Ee3b\nb52ppmXEvts2ps8BW3HuM6NZvmrtuvKZC5axZ/vwB2TtGvl0ab0pk/63lHmLVzJr4XK2bFIXgL22\nblxpb66J1D9AAWBm/c3slJjX/zaz1mZW18yONbM/Yur+MLPuUV1rM/t3ea8r3TneL4FTJXUk5E96\nAfsAN0f1wwlB9p7o58houzCmbIqZTVdI1EwEXpT0NPCFmc0p7c0l9QH6ALRs1TrFl1ays04/mc+G\nf8r8+fPo1L4N1/TtxwfvvcOECePJy8ujVavW3Pvg+vHaQ98czAEHHULdunXT1sZsUVhYyEcffsDD\njz6xruyRx57iqssvYfXq1dSsVYuHHyvuhnVuuP+ULuzerhEN69Zg5PUH8MB7Ezj/oHbUqJbHc+eG\nIXTfT1vI9a/+xIufTeOOE3fknau6IcKIiHGzwj2fG18fy30nd6F6vvj9jzAErbLKxUeGZWbpeSNp\nKnA2sAdQF/gUuAI4nNDb3ZKQ/H4VaAzcD/xE6NpPAraIyhqY2RnROVsSBjEfHh0/EjgrJgleop12\n7moff/ZVyq4vl9Sqnp94pyqq0z/eznQTstb0ly5hxewJKQ2T23fe2V55N/EfsR2b1x1tZl1T+d4V\nKRPDyV4A/gb0Zn2aociXhLuN2xN6tyOip0V+jykryu9iZtPN7EIzawe0AZYWc07nXCVWEamGTEt7\n4DWzaYSbbEcQ7hbG1i0nPEN9FNDMzH6NqkZEZTsSE3jjjv2d8Ez19hXTcudcJqRmUEN2ydQDFGcB\nB0ZDNeINBy4l5hE+QgrhUmC2mU0CkNRQ0o2S2kvKi262nUnoNTvnckUORt6MBF4zm2Rmo0qo/pQw\nNCw2sTMyKovt7a4k5IQ/JAwh+wlYQUhhOOdygJSaJ9eyTdpGNZhZ2xLKVxPznWVm7xH3HRaNVogv\nW0oYjuacy2GVL6wm5o8MO+eyWw5GXg+8zrksVqaJzrOeB17nXFarhCnchDzwOueyVtEjw7nGA69z\nLqt5qsE559LMe7zOOZdmORh3PfA657KYKHZZrMrOA69zLmvl6s01X+zSOZfVUjFVg6Sakv5P0jRJ\niyV9J+nwmPqDJP0qqVDSx5LaxB37TLQM2WxJl5f3mjzwOueyWoqmhaxGmF52P2AT4HrgZUltowm2\nXovKGgGjCCvfFOkPbE2YevYA4GpJh5XnmjzV4JzLaqkYThbN7dI/pugtSVOAXYDNgLFmNghAUn9g\nnqQO0dS0pwFnmNkCYIGkpwiTcb1b1vZ4j9c5l9UqYiJ0SZsT1m8cC3QCxhTVRUF6EtBJUkOgeWx9\n9Hunsl+R93idc1lsIwJrY0mxU80+aWbFLs4nqTphReHnzOxXSfWAuXG7FRBWG64X8zq+rsw88Drn\nslqSqYZ5yay5JimPsPzYSsIiuhCWem8Qt2sDYHFUV/R6eVxdmXmqwTmX3VK0AkW0Mvn/AZsDPc1s\nVVQ1Fugcs19doB0h77sAmBVbH/0+tszXgwde51yWS+HKP48BHYGjzWxZTPnrwPaSekqqBdwA/BCz\n5uPzQN9oubEOwDnAgHJckgde51w2S7zsTzJL/0Tjcs8FugCzJS2JtpPNbC7QE7gFWADsDpwYc3g/\nws22aYSlye4yszKPaADP8TrnsliqnlyLVjcv8Uxm9iHQoYS6FYSFdM8sf0sC7/E651yaeY/XOZfV\ncnGuBg+8zrms5hOhO+dcGkmQl3tx1wOvcy7LeeB1zrn08lSDc86lmd9cc865NMvBuOuB1zmX3XzN\nNeecS6NcXXNNZpbpNmSEpLmEZ6+zQWNgXqYbkaX8syldNn0+bcysSSpPKOldwjUmMs/MyrUcTzpV\n2cCbTSSNSmYu0arIP5vS+edTOflcDc45l2YeeJ1zLs088GaHYteGcoB/Non451MJeY7XOefSzHu8\nzjmXZh54nXMuzTzwOudcmnngzULKxWcknXPreODNQuZ3PF0ZSTo6WoLcZTGfqyGLSDoH2AL4Chhl\nZn9kuEmuEpE0EGgC3CJpSrQ6rstCPpwsS0gaDLQAlgBNgQfM7ElJ8h6wSyQKus2BI4FlZrYqrj7P\nzNZmpHHuTzzVkAUkDQFqm9muZnYA8ClwrqRqVT3oSjpOUqdMtyObSeoKtAYOM7NFZrZKUhNJ+0k6\nBMDM1kryf+9ZwlMNGSbpAEIvZf+Y4jcIvd+6QEEGmpUVJG0P3AS8Ken/zGx8ptuUpQyob2ZLJTUF\ndgaeBxYADSUNN7PjvMebPfwbMMPM7GOgH/CepC6SqgNPAR+ZWZUNugBm9hNwGrAHcJakbYrqJOXH\n/L5JBpqXcZIuk3SmmY0Gakn6AXgWuA94CNgP+BuwW1HP12UH7/FmUFHezcxuiv4MHAUsA641s4ei\nfapcjlfSrUA7YC5wP3AJIZCYpGfNbJyZrYn2vQToLOk8M1uZsUanmaS6wO6EztMzhF7uzcAYYIqZ\nfRTtV0iYd3pxhprqiuE31zJA0i7AeDNbHFd+EfAAsL2Z/SypevxNklwn6Q3CxNfDgC6EwNKd0Hvr\nB3wBPGNm4yRdCdwJ7GJm32WoyRkjqR0wGrjLzG4pYZ9zgUuBg81sRjrb50rmPd40k3QzcC0wUtIt\nwA9mNgvAzB6S1AT4XtKhZvZxVbobLekVoEXRxN6SugO3A83MbJgkA/oDC6Nc5nnArlUp6Eo6D3ga\nyDOzSZLOB/pL+qKolxvttw+wN3AV8BcPutnFA2/6/QRMAf4ABgDfSfoIeBhYYWY3SFoDDJO0r5mN\nzFxT00dSI+BQ4AFJdcysEPgZKATqRSmXj6KH+h4k3HzcJ8pvVgmSegCPEnLeH0oaDgwifG6HSxpt\nZgVRGuLwaL8DzOzHjDXaFctTDWkiqZaZLZe0FfAxcCXwLXAK0JuQz3yH8GfjEkkXAsPM7JdMtTld\nJN1ASCFMB94kfCG9AwwGHjazO+P270pYY2tqeluaWZJaET6TpYSAezihR9uS8MXd28w+i/atTRii\n6A/hZCEf1ZAGkp4CTgYws8nA3YTc5Fozu5EQhHcBjgNmSboceKyKBN2GQG3gFqARcCxwJvA58FJR\n0I0dg2pmo6pa0AUws9+Bi4B8YDLwMjAEqAHMBB6RVD/ad5kH3ezlgbeCRTeLdjKz/4sp/hiYA7SW\ntC9h+M8FwA6E/O/gorv2uc7MFgCPEMYu3ws0BP4CzCDkchtF+1WJPHc8SddL+kJS+ygF8znwFrCd\nmQ0ALgQOJqw0vCPh/0cuy3mqoQJJeh1oYmb7xJTVMLOVkh4FTiU8InydmT2TqXZmQvRU3uqY19WB\nW4GjgDOARYTe3H8Ij0/PzUhDM0jSPwhPMd4CVAc+BB4jzOdxO3CZmY2X1BrYmjCc7Awz+zVDTXZJ\n8sBbQSQ9Tbjp0TZmzOlWhBtDFwIrgA+A4WZ2QVUaryvpNsJNxleAlWZmkpoTct4/AfWBKwi9uC8I\nPeFbqlKvN8rxPwi8D/QBjgYOAjoCJxIeLNkL2K9o/HJVGgFT2XmqoeK8QLgj/xcASS2B4cD3UX5y\nIfAdofdS1aaC/Bi4ATg6CrqtCDOyPUD4shpMyIE3Izwk8J8qGFCGAiOA3QiphaGE4XNDCcH4D0Ja\n5ryi+Zur4GdUaXngrSBm9imhp3JvNPbyC+BRM+sb9W6XEYLNAZI2rUqTn5vZ+8A5wE3RZ/M58LiZ\n3Rb9dfBv4COgL/C7mU3MXGvTL3oceiYhyJ5O+IIeDtQ1sysJfzF1BLYBegC1MtRUV0aeaqhg0c2z\nt4G3zOzEmPLLCH8unmhm4zLVvkwq7rMp+nM5Sj2sMLP5GW1kGknaE/g6JjV1BnAZodf7BOEm2j5m\nNkVSC8I43Z+q6v9/KjMPvGkgaW/C00bXmdlrUS/vNsJjnFXmAYDiRJ/NU8B1wHvRgxNVjqQBhC/i\npwkjXvpFX0C3A7PM7AGFOZt3JDyJVqX+Csg1HnjTRNJ+hIlevidMA3mImX2b2VZlh+izeZRwV/7V\nqjTZTRFJexFyt88DWwIC3iPkuWuY2aXRfh8Q7gvsFDsqxFUuHnjTKJp799+ECavHZLo92UTSwYS/\nAg6Mnzyoqoh6/48DlxMmNq9PyHM3Ag41sw+i/Vqa2fSMNdSVmwfeNCt6dDjT7chGMXM0VFnRl/OD\nwA1m9rqkLkB7wiPUy3zkQm7wwOtclpG0PyHv/U8zeyXDzXEVwGcncy7LmNknks4GHoue6KuSee9c\n5j1e57KU571zlwde57KY571zkwde55xLM39k2Dnn0swDr3POpZkHXuecSzMPvM45l2YeeJ1zLs08\n8Loyk/STpP4xr6dKujID7egqySS1rcD36C/ppxScJyOfkcsuHnhziKQBUQAySaskTZZ0t6S6aWrC\nroRZxhKS1FvSkgpuT+z7fSLp4XS9n3Ol8UeGc8+HhEU0qwPdCPO71gXOL25nSdXNbFUq3rgqLkjp\nXFl4jzf3rDCz2Wb2u5n9G3gJ6A5h8pWoN3yEpK8lrSSscYakoyWNlrRc0hRJt0iqUXRSSU0lvSFp\nmaRpks6Mf+P4P6MlNZD0mKRZ0Xl/kdQrmgTmWaBuTA+9f3RMDUl3SJouaamkbyQdGvc+h0n6NTrn\nCMISOOUi6XZJ46LrmyrpTkl/WlJH0tmSfov2GyypcVz9GZJ+jto2XtJlkvzfmduA93hz3zJC7zfW\nHYRVfCcCi6PA9hJwCWFtr9aEeWFrAkWBdADQhrD8TCFwH9C2pDeN1pB7h7Ag4xnAeGBbwvpgnwOX\nEpZzbxcdUpR2eDYq+xswHTgCGCJpVzMbEy2MOZgwe9cjhBUZ7k3+4yjRUuBMYAawHeH6VwDXx+zT\nFjgFOBaoAzwJPAMcE13zOcC/gIuA0cD2UTtXAZ7mcOuZmW85shGC41sxr3cjLJE+MHq9P2BAz7jj\nhgPXx5V1JwRDEXqUBuwdU98GWAP0jymbClwZ/X4IsBboWEJbewNL4sraRce0jisfTFgoFEKwHk/0\nuHtU1jdqX9tSPptPgIc34rM8D5gY87p/dL2tY8r2id536+j1b8Cpcee5FPi5uM/It6q7eY839xwW\n3bSqRujpvkHogcUaFfd6F2A3Sf+IKcsDahOWmelICIhfF1Wa2TRJM0tpx06EtcJ+2Yi270wI9D/H\nLbpck7DqMFFbvjSz2ElGvtiI9yiWpOMIQbI9UA/Ij7ZYM8zst5jXXxF9uUhaCLQCnpD0WMw+1QjX\n5Nw6Hnhzz3DCsvKrgJlW/I2zpXGv84AbgUHF7DuXsgWOshyTR+hB7kpof6xl5ThvqSTtAfyX8Blc\nBiwkpA/u3ojTFOVxi5ard65EHnhzT6Ft/Aq03wIdSjpO0i+EwLIrUVCR1BponuCczSR1LKHXu5I/\n9yi/IwTWLczs4xLO+zPQU5Jier17lNKOZOxN6M3eVFQgqU0x+7WQ1MrMfo9e70b4XH4xszmSZgDt\nzOz5crbH5TgPvA7CDaG3JE0DXgZWE24M7WZmV5vZOEnvEv6M7kPofd7L+l5ocYYR/hR/VdJlhLxs\ne6CumQ0m5DprSTqEEHALzWy8pJeAAZKuIATvRoTc9GQze41w0+sK4H5JjwI7EHqZyWgcrWEW639R\n21pIOpmQtjgUOKmY45cBz0m6nJCGeRwYamYTovr+wENR2uFtQqpnZ6CFmd2WZBtdFeDDXBxm9h5h\nyfkDCHncr4FrCDeLivQGphByrUMIqyVPLeWca4HDgc+AF4FfgAeAGlH954TA9R9COuPq6NAzCCMb\n7gR+Bd4C9gWmRcf9BvQADgPGEFID1yR5qb0IQT52u9zMhgB3AfcDPxBuDN5QzPFTCSmJIdHnMDlq\nb9E1P00YGXFq1LYRhLTPlCTb56oInwjdOefSzHu8zjmXZh54nXMuzTzwOudcmnngdc65NPPA65xz\naeaB1znn0swDr3POpZkHXuecS7P/BxJ3pwFWSGCTAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2a3a76fb978>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Confusion Matrix\n",
    "conf_mx = confusion_matrix(y_test, y_pred)\n",
    "plot_conf_matrix(conf_mx, train.author.unique(), title=\"SGD Pipe Confusion Matrix\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 273,
   "metadata": {},
   "outputs": [],
   "source": [
    "parameters = {\n",
    "    'vect__max_df': (0.5, 0.75, 1.0),\n",
    "    'vect__max_features': (None, 5000, 10000, 50000),\n",
    "    'vect__ngram_range': ((1, 1), (1, 2)),  # unigrams or bigrams\n",
    "    'tfidf__use_idf': (True, False),\n",
    "    'tfidf__norm': ('l1', 'l2'),\n",
    "    'clf__tol': (1e-2, 1e-3, 1e-4),\n",
    "    'clf__C': (1, 10, 100, 1000),\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 274,
   "metadata": {},
   "outputs": [],
   "source": [
    "svc_clf = Pipeline([\n",
    "    ('vect', CountVectorizer()),\n",
    "    ('tfidf', TfidfTransformer()),\n",
    "    ('clf', LinearSVC())\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 275,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(14684,) (4895,) (14684,) (4895,)\n"
     ]
    }
   ],
   "source": [
    "print(X_train.shape, X_test.shape, y_train.shape, y_test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 276,
   "metadata": {},
   "outputs": [],
   "source": [
    "svc_clf_gs = GridSearchCV(svc_clf, parameters, n_jobs=-1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 277,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=None, error_score='raise',\n",
       "       estimator=Pipeline(memory=None,\n",
       "     steps=[('vect', CountVectorizer(analyzer='word', binary=False, decode_error='strict',\n",
       "        dtype=<class 'numpy.int64'>, encoding='utf-8', input='content',\n",
       "        lowercase=True, max_df=1.0, max_features=None, min_df=1,\n",
       "        ngram_range=(1, 1), preprocessor=None, stop_words=None,\n",
       "        strip...ax_iter=1000,\n",
       "     multi_class='ovr', penalty='l2', random_state=None, tol=0.0001,\n",
       "     verbose=0))]),\n",
       "       fit_params=None, iid=True, n_jobs=-1,\n",
       "       param_grid={'vect__max_df': (0.5, 0.75, 1.0), 'vect__max_features': (None, 5000, 10000, 50000), 'vect__ngram_range': ((1, 1), (1, 2)), 'tfidf__use_idf': (True, False), 'tfidf__norm': ('l1', 'l2'), 'clf__tol': (0.01, 0.001, 0.0001), 'clf__C': (1, 10, 100, 1000)},\n",
       "       pre_dispatch='2*n_jobs', refit=True, return_train_score='warn',\n",
       "       scoring=None, verbose=0)"
      ]
     },
     "execution_count": 277,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "svc_clf_gs.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 278,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.81803323345137569"
      ]
     },
     "execution_count": 278,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "svc_clf_gs.best_score_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 279,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "clf__C: 1000\n",
      "clf__tol: 0.01\n",
      "tfidf__norm: 'l2'\n",
      "tfidf__use_idf: True\n",
      "vect__max_df: 0.75\n",
      "vect__max_features: None\n",
      "vect__ngram_range: (1, 2)\n"
     ]
    }
   ],
   "source": [
    "for param_name in sorted(parameters.keys()):\n",
    "    print(\"%s: %r\" % (param_name, svc_clf_gs.best_params_[param_name]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 284,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = mnb_gs_clf.best_estimator_.predict(X_test)\n",
    "y_pred_prob = mnb_gs_clf.best_estimator_.predict_proba(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 285,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy score: 0.862717058223\n",
      "Log loss: 0.354775708031\n"
     ]
    }
   ],
   "source": [
    "print(\"Accuracy score:\", accuracy_score(y_test, y_pred))\n",
    "print(\"Log loss:\", log_loss(y_test, y_pred_prob))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 286,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV4AAAEqCAYAAABdmDjNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XeYVEXWx/Hvb0hKkqxkFEUQFFQw\nBxRFMPKqu2JAQQVd15zWNeKa0645oCLmtKsgiFmCBAOIIBhQBFSCZCTD4Hn/qDvQtBOamZ7unp7z\n8bkP3bduOLeF09V161bJzHDOOZc6OekOwDnnyhtPvM45l2KeeJ1zLsU88TrnXIp54nXOuRTzxOuc\ncynmibcckvSOpLPSHcfWkvQ3Sb9JWimpbgmOs1LSTsmMLdUknS7p/XTH4YrHE2+KSDpI0jhJyyUt\nkTRWUidJ+0taJalGPvtMknRh9LqypP6Sfoi2nyVpoKQWBZxvlqQ1UZL5TdIzkqoDmFl3M3u2lK6z\nlaTXJS2KrnWKpMslVSjhcSsB/wa6mll1M1tc3GNF+/9UknjyE33m6yXVi1v/lSQr6P9V3LYtom0r\nFradmb1oZl1LFrFLF0+8KSCpJjAMeAioAzQGbgbWmdl44FfgpLh92gG7AS9Hq/4LHA+cBmwHtAcm\nAl0KOfVxZlYd2AvoBFyfpEvKl6SWwGfAL8DuZrYd8BegI/CnL5attD2wDTCthMcpbTOBU/PeSNod\n2DaZJygqKbsywMx8KeWFkHiWFVJ+LfBx3Lq7gTei10cAa4CmW3HOWcARMe/vAYZFr0cC50avewNj\nCV8Ky4HvgC4x+20HPA3MA+YAtwIVCjjnC8DbRcR1PCF5LoviaBMX85XAlCiWVwnJthWwCjBgJfAx\n0CJ6XzFm/9jr2hkYFR1nEfBqzHYG7Bxzfc8BC4HZhC+nnJjPZgxwL7CUkFS7F/GZXw98EbPuXuC6\n6JwtonXHAJOA3wlfUv1jtv855jpXAvvH/D/6D7Ak+n/QGxgT7XNAdI1No/fto8+3dbr/7vuS/+I1\n3tSYDmyU9Kyk7pJqx5U/DxwsqRmApBxCzfa5qPwI4HMz+6U4J5fUFDia8I89P/sCPwH1gJuANyTV\nicqeBXIJiWxPoCtwbgHHOYJQMy8ojlaEGvylQH1gODBUUuWYzf4KdAN2BPYAepvZdKBtVF7LzA4v\n8GI3uwV4H6gNNCF8seTnIULy3Qk4FDgT6BNTvi/wPeGzuRt4WpIKOe+nQE1JbaLmlVMIX0ixVkXn\nqUVIwn+T1CMqOyT6s5aFJpHxMXH8BDQAbos9mJmNA54AnpW0LeHv0/Vm9l0hcbo08sSbAmb2O3AQ\noSbzJLBQ0luSto/KfyHUzs6IdulCqOm9Hb2vS6hxbq3BkpYRam2jgNsL2G4BcL+ZbTCzVwmJ5pgo\nvu7ApWa2yswWEGpdPQs4TlFxnkKoEX9gZhsItcFtCTW2PA+a2VwzWwIMBTokdKV/tgFoDjQys7Vm\nNiZ+g5jE+E8zW2Fms4D7gF4xm802syfNbCPhS6ghodmjMM8TEuuRhF8Qc2ILzWykmX1tZn+Y2RTC\nl9GhRRxzrpk9ZGa5ZrYmn/L+hC+Qz4G5wCNFHM+lkSfeFDGzb82st5k1AdoBjYD7YzZ5lvCPFcI/\n/Jei5ASwmPAPfmv1MLNaZtbczC4o4B8swBwzix0taXYUX3OgEjBP0rIoiT9BqHXlp6g4G0XHBsDM\n/iD81G4cs838mNergeqFHK8wVwMCPpc0TdLZ+WxTD6gcG1P0Ot94zGx19LKomJ4n/GLpzeZfLZtI\n2lfSCEkLJS0Hzo9iKUyhv3aivyuDCH+37ov7/+kyjCfeNIh+Ag4i/CPJ8wbQWNJhwIls+Q/2Q2Af\nSU1KKaTGcT+fmxFqTb8A64B6UQKvZWY1zaxtvkcJcZ5UQBnRMZvnvYnO2ZS4GmGCVkV/Vo1Zt0Pe\nCzObb2Z9zawRcB7wqKSd446xiM014zzNihnPJmY2m9AefDTh/2u8l4C3CG2y2wGPE74kIPwqyvew\nhZ1TUmNCM9EzwH2SqhQjdJcinnhTQFJrSVfkJc6ozfVUQnsgAGa2itA++gzh5+2EmLIPgQ+ANyXt\nLamipBqSzi+gJre1GgAXS6ok6S9AG2C4mc0jtJPeJ6mmpBxJLSUV9LP4JuAASfdI2iG61p0lvSCp\nFvAaoQmjS9Q97ApCYh+3tQGb2UJCgjxDUoXoc2iZVy7pLzFfVEsJiWtj3DE2RjHdFn2ezYHL+XOb\nbHGcAxwe/X+NVwNYYmZrJe1DqB3nWQj8QWhzTkj0BTaIcBP0HEJzzy3FjNulgCfe1FhBuDnymaRV\nhIQ7lZB4Yj1LqH396ecpcDLhZtSrhDv1Uwm9JT5MQnyfAbsQaoC3ASfb5n6yZxJ+jn9DSGD/pYDm\nBDObQbgL3wKYFv2M/h8wAVhhZt8T2rEfis51HKHL2/pixt0XuIrQxNGWLRN4J8LnvZJQu7zEzGbm\nc4yLCLXnnwht4S8BA4sZzyZmNiP2yzPOBcC/JK0AbiQk/7z9VhP+H4yNmnf2S+B0FxPanW+Imhj6\nAH0kHVyii3ClRt4UVL5J6k3ognVQumNxrrzwGq9zzqWYJ17nnEsxb2pwzrkU8xqvc86lWLkdbEMV\ntzVVLum4LdmpfZtm6Q4hY/kvxIL98vNsFi9aVNjj1FutQs3mZrkFPfezma1Z+J6ZdUvmuUtT+U28\nlWtQZde/pjuMjDRq7IPpDiFjrc/9I90hZKwjDtk36ce03DUJ/Ttd+9UjRT35l1HKbeJ1zpUFAmVf\ni6gnXudc5hKQU6Ix9DOSJ17nXGYrdBTOsskTr3Mug3lTg3POpZ7XeJ1zLoWE13idcy615DfXnHMu\n5bypwTnnUslvrjnnXGoJr/E651zKeY3XOedSSVAh+26uZd9XiXMue+R1JytqSeRQ0oWSJkhaJ2lQ\nXFlVSY9KWiRpuaTRMWWSdJekxdFyd+ys3JI6SJooaXX0Z4eiYvHE65zLbFLRS2LmAreS/2SmA4A6\nhBm26wCXxZT1A3oA7YE9gGOB80JoqgwMIcxMXZswYe2QaH2BPPE65zKYklbjNbM3zGwwYVbqzWeQ\ndgWOB/qZ2UIz22hmE2M2OQu4z8x+NbM5wH1A76isM6HJ9n4zW2dmD4agObywWDzxOucyW2I13npR\nM0Le0m8rzrAvMBu4OWpq+FrSSTHlbYHJMe8nR+vyyqbYliPkT4kpz5ffXHPOZS4l/OTaIjPrWMyz\nNAHaAf8DGgH7A29L+sbMvgWqA8tjtl8OVI/aeePL8soLnd7Ga7zOucyWpKaGQqwBNgC3mtl6MxsF\njAC6RuUrgZox29cEVka13PiyvPIVhZ3QE69zLrMl7+ZaQaYUUT6NcGMtT/toXV7ZHrG9HAg34KZR\nCE+8zrkMlryba5IqStoGqABUkLSNpIrAaOBn4J/RNgcSbpq9F+36HHC5pMaSGgFXAIOispHARuBi\nSVUkXRit/7iwWDzxOucyW/JqvNcTmhWuAc6IXl9vZhuAE4CjCe2zTwJnmtl30X5PAEOBr4GpwNvR\nOsxsPaGr2ZnAMuBsoEe0vkB+c805l7kkyElOmjKz/kD/AsqmEW6q5VdmwNXRkl/5JGDvrYnFE69z\nLrP5IDnOOZdiPkiOc86lmNd4nXMuheQDoTvnXMopJ/sSb/ZdUYZ4/KbTmf3RHUx4/dpN656/sw+f\nvnINn75yDd+9fTOfvnINAIfv25qxL17NF69dy9gXr+bQTq027XNy1734/NV/MvG/13HbJSek/DpK\n29/PO4eWzXZgv7332LRuyZIlnHBMV/ZstysnHNOVpUuXAmBmXH35JXRo24oDOnXgq0lfpivslLj4\nb+fSZsdGHLzP5lEG7779X+zeqjmdD9ibzgfszQfvvbPFPr/+8jPNd6jFIw/8O9XhloowAYWKXMoa\nT7yl5Pmhn3LC3x/ZYl2va55hv553sl/POxn80VcM+fgrABYvW8nJlz5Bp7/eTt8bn2fgrWcCUGe7\natx+aQ+OPv8h9j75NhrUrUnnfVr96Vxl2Wm9zuJ/Q4Zvse4/997FoZ27MGnq9xzauQv/ufcuAD54\n7x1mzPiBSVO/54GHH+fyi/+ejpBTpufpZ/HKm8P+tP78v1/CyHETGTluIkce1X2LsuuvuZIuR3ZL\nVYilTwkuZYwn3lIy9ssZLFm+usDyk47ci9feDSPPTf7+V+YtDONsfDNjHlUqV6JypYrs2LguP/y8\ngEVLVwLw8Wff0aNLkWMslykHHnQItevU2WLd8GFvcdoZ4cvntDPO5O2hQwB4e9hbnHpaLyTRad/9\nWL58GfPnzUt5zKlywEEHU7t2naI3jAwfOoQWLXakdZvdSjGqVCu6tus1XpeQA/dqyW9LVjDj54V/\nKvu/Izow+ftfWL8hlxm/LGTXFtvTrGEdKlTI4fjD2tNk+9ppiDi1Fi74jR0aNgRgh4YNWbhwAQDz\n5s6hcZOmm7Zr1LgJc+fOSUuM6fT0gEc5dL89ufhv57IsaoZZtWoVD/3nHq785w1pji75PPGWEkmz\nJK2RtDJmeTimvLMkk3R13H4tovV5+8ySdE3qr2Dr/LVbR15/d8Kf1rfZaQduvfgELrz1FQCWrVjD\nxbe/ygt3nc1HAy9j9tzFbNz4R6rDzRhbDnkalMV/dCXR+9zz+GLK94wYN5Htd2jIjddeBcDdt93M\neRdeQvXq1dMcYfLl5OQUuZQ1mdSr4Tgz+7CAsrOAJdGfd+dTXsvMciXtD3wk6Ssze7e0Ai2JChVy\nOOHw9hx42paX0bhBLV79dz/OveF5Zv66aNP64aOnMnz0VADOPvHAcpF46zfYnvnz5rFDw4bMnzeP\n+vUbAKGGO+fXXzZtN3fOrzRs2ChdYaZFgwbbb3rdq/c5nP6XHgBMnPA5Q4e8wb9u+CfLly8jJyeH\nKttU4dzzyng7eBltwy1Kxn9VSKoKnAz8HdhFUoGDHZvZeMJwbO1SFN5WO3zfXZk+6zfmLFi2ad12\n1bfljYfO58aH3mL85J+22L5+7VCDqVVjW/r99WCeeXN8SuNNh+7HHMdLLzwHwEsvPMfRxx4PwNHH\nHMfLLz2PmfHFZ59Ss+Z2m5okyov58ze3aQ8fOpjWu4WJDoa9P5Ivp/3Il9N+5LwLLubSK64p+0kX\nUJa28WZSjbcgJxEGG34dOJ0wCtCffqdH42EeQJhyY1J+B4qmAwlTglQq3Z9kz97Rm4P33oV6tarz\n47u3cMvjw3l28Hj+ctTem26q5Tm/5yG0bFqfa/p245q+4Y70cX97mIVLV3Lv1Seze6vGANwx4F1+\n/HlBqcadamefeRpjPhnF4kWLaNOyGf+84SYuv/IfnHVGT55/diBNmjbj2RdfBaBrt6N5/7136NC2\nFVWrVuWRJ55Oc/Slq1+fMxj7ySiWLF7EHru24Oprb2TcmFFMnTIZSTRt1oJ7H3w03WGWurKYWIui\n/NrNUh6ENAuoB+TGrL7KzJ6U9CEw1cwulXQq8CDQyMw2SGoBzCQM5WbAfOCxaMK5QuVUbWBVdv1r\nci8kS/w2vsiPr9xan5v9TT3FdcQh+/LVlxOTmiUr1t3Jah59a5HbLX3h9IklmPon5TKpxtsjvo1X\nUlPgMOCf0aohhGmYjwEGx2xaz8xik7ZzLhsIlJN9Nd5Mb+PtRYhxqKT5wE/ANoTmBudcOZCsNl5J\nF0YzEK+TNKiAbW6KekodEbOuiqSBkn6XNF/S5XH7dJH0naTVkkZIal5ULJmeeM8EbgY6xCwnAcdI\nqpvOwJxzpS/JN9fmArcCA/M9l9SScCM//qmc/sAuQHPCL/CrJXWL9qkHvAHcANQh3H96tahAMinx\nDo3rxzsCaAE8YmbzY5a3gB+BU9MarXMuJZKVeM3sDTMbDCwuYJOHgX8A8dP2nAncYmZLo+nenwR6\nR2UnAtPM7HUzW0tI0u0ltS4sloxo4zWzFlu5fduYt9nXAOSc2yyxf+H1JMX2dhpgZgMSPoX0F2C9\nmQ2PTeSSagONgMkxm08mzLMGoRfVpjIzWyVpRrT+OwqQEYnXOefyJRJ9Mm1RcXs1SKoO3A50zac4\nr9/p8ph1y4EaMeXxz/7HlufLE69zLqOloB/vzcDzZjYzn7KV0Z81gbUxr1fElNeM2ye2PF+Z1Mbr\nnHNbSNGTa12Ai6MeC/OBpsBrkv5hZksJN9vax2zfnvCELNGfm8okVQNaxpTnyxOvcy6zJWk8XkkV\nJW0DVAAqSNpGUkVC4m3H5p5Tc4HzgLwBtZ8DrpdUO7pp1hcYFJW9CbSTdFJ07BuBKWZWYPsueOJ1\nzmUyJXVYyOuBNcA1wBnR6+vNbHFszylgI7DUzPKaGW4CZgCzgVHAPXmDcJnZQkIX19uApcC+QM+i\nAvE2XudcRkvWsI9m1p/Q3auo7VrEvV8HnB0t+W3/IVBo97F4nnidc5ktCzuMeuJ1zmW0bBydzBOv\ncy5jldXxdoviidc5l9E88TrnXIpl47CQnnidcxnNa7zOOZdK8sTrnHMpJSAL864nXudcJvNeDc45\nl3I5fnPNOedSSN7U4JxzKSW8xuuccynnNV7nnEsxv7nmnHMpJGVnU4MPhO6cy2DJm/pH0oWSJkha\nJ2lQzPr9JH0gaYmkhZJel9QwplyS7pK0OFruVsxJJXWQNFHS6ujPDkXF4onXOZfRpKKXBM0FbgUG\nxq2vDQwAWgDNCRNVPhNT3o8wnXt7YA/gWMLUQEiqDAwBXoiO8ywwJFpfIE+8zrmMlqwar5m9YWaD\ngcVx698xs9fN7HczWw08DBwYs8lZwH1m9quZzQHuA3pHZZ0JTbb3m9k6M3uQ0Bnj8MJi8cTrnMtc\nCdR2S+He2yFsOUtwW2ByzPvJ0bq8silmZjHlU2LK8+U315xzGWsr+vHWkzQh5v0AMxuw1eeT9iDM\nFHxCzOrqwPKY98uB6lE7b3xZXnmNws7jidc5l9ESbEpYZGYdS3ienYF3gEvM7JOYopVAzZj3NYGV\nZmaS4svyylcUdi5vanDOZbRUNDVIag58CNxiZs/HFU8j3FjL057NTRHTgD205bfDHmzZVPEn5bbG\nu0frprw/6j/pDiMjdblvdLpDyFijrjo03SFkrJzSeNAhiePxSqpIyHkVgAqStgFyge2Bj4FHzOzx\nfHZ9Drhc0nDAgCuAh6KykcBG4GJJjwN9o/UfFxZLgYlXUtVELyi6E+icc0mV5PF4rwduinl/BnAz\nIZnuBNwkaVO5mVWPXj4RlX8dvX8qWoeZrZfUI1p3J/At0MPM1hcWSGE13pVRQImokOB2zjm3FZS0\nJ9fMrD/Qv4DimwvZz4CroyW/8knA3lsTS2GJt/vWHMg550pDuRqrwczeS2Ugzjn3J1k6Hm/CvRok\n1Y2edf6PpLrRuk6SmpZeeM658iy08SbnybVMklCvBkntgY+AhUBLwh29xcDxhOebe5VSfM65cq4s\nJtaiJFrjvQ94yszaAOti1r8DHJz0qJxzLpKToyKXsibRfrydiEbjiTMH2CF54TjnXIwsbeNNNPGu\n48+PxQG0AhYlLxznnNtMWTq9e6JNDUOB66MnPwBMUmNCh+E3SyUy55wjLaOTlbpEE++VQBNgPrAt\n4XG4n4D1wLWlE5pzzoVHkYtaypqEmhrMbKmk/YFuwF6EhP0lMNzM/ijF+Jxz5Vi2zrmW8CA5UYId\nHi3OOZcSWZh3t+oBim6S3pf0a7R8IMkfK3bOlapsfIAiocQr6SLgLeA34PZomQ8MlnRx6YXnnCvv\nsvHmWqJNDdcAl5nZIzHrHpU0njDU2oNJj8w5V+6J0KUs2yTa1LAdkN+gOe+Tf/9e55wrOYkKOUUv\nZc3W9OM9IZ/1J+A325xzpSgbmxoKTLySLshbCCOvXy9psKRromUwoZlhUqqCdc6VLyJ5/Xij0RUn\nSFonaVBcWRdJ30laLWlENAdbXlkVSQMl/S5pvqTLE923IIW18d4Q934tsG+0xK67CLijqBM551xx\nJLFGOxe4FTiK8CBYdHzVA94AziX8ur8FeBXYL9qkP7AL0JwwNs0ISd+Y2bsJ7JuvwgZCb1iMC3PO\nuaRKVncxM3sjOl5HwpO4eU4EppnZ61F5f2CRpNZm9h1wJtDHzJYCSyU9CfQG3k1g33z59O7OuYwl\nkejNtXpRM0Le0m8rTtMWmJz3xsxWATOAtpJqA41iy6PXbYvat7ATJvzkmqQWhOzeDKgcW2ZmFyR6\nHOec2xoJ1ncXmVnHYp6iOmGSh1jLgRpRWd77+LKi9i1QojNQHEl4gOI7Nmf4nQizC3+eyDGcc644\nUvBk2kr+3C22JrAiKst7vzaurKh9C5RoU8PtwJ1mtidhbN5TCDXfUYSE7JxzSRd6NRS9lNA0oP2m\nc0rVCFOcTYvadefFlkevpxW1b2EnTDTxtgZeiF7nAttGbRk3AlcleAznnNs6CYzTkGiNWFJFSdsQ\nfqlXkLRNNMb4m0A7SSdF5TcCU2Jujj1H6E5bW1JroC8wKCorat98JZp4V7G5XXceIaMDGFA3wWM4\n59xWS+Kca9cDawhDIJwRvb7ezBYCJwG3AUsJXWZ7xux3E+GG2WzCr/x7zOxdgAT2zVeiN9c+Bw4A\nviV0obhHUpvohN7G65wrFXlNDclgZv0JfXLzK/uQ8Ms+v7J1wNnRslX7FiTRGu9VwFfR65uAccA5\nwILoT1eISy7oy247NeaQfTtssf6pxx/hgL3acsg+7fnXDddsWj9t6hSO7nIwh+zTnkP325O1a9fG\nH7JMu+HYXXnv0gN4pW+nTeu6tK7Pq/068dm1h9Km4eYbwt3aNuDFcztuWj679lBabR9uNB/Zpj4v\nnduRV/t14qLDd0r5daTasmXLOL3nX9hz9zbstcdufPbpeAAee+QhOrRrTccO7bjun1enOcrky8Zh\nIROdgeL7mNcrgD4AkioBTUsntOzR8/QzOaffBVx4Xp9N68aMHsm7w4cyYvyXVKlShYULFwCQm5vL\n3/v25pEBz9B29/YsWbyYSpUqpSv0UjFs8nxemzCHm49rs2ndjIWruPq/U/nn0btuse270xbw7rTw\n2bSsX437/tKO6b+tZLttK3Jxl5b0GjiRZas3cNNxrenUohZfzFqW0mtJpauuuJQjux7Fi6+8zvr1\n61m9ejWjRo5g2NC3+GziZKpUqcKCBQvSHWbSlb20WrSSPkCxG/BDMgLJZvsfeDC1atfeYt2zTz/B\nRZddRZUqVQCoX78BACM/+oDd2u5O293DjdI6detSoUKF1AZcyib9spzf1+RusW7W4tXMXrKm0P2O\natuA974JiaVxrW35eckalq3eAMDnM5dyeOv6pRNwBvj9998Z+8lozuoTfmBWrlyZWrVq8dSAx7ni\nqn9s+nvUoEGDdIaZdFJ2zrnmT66lyYwff+CzcWPodtiB9OjehUkTJ2xaL4lTehzDEQfvw8P335vm\nSDPHkbs14P2o9vvL0jU0r1uVhtttQwWJzrvWY/uaVdIcYemZOfMn6tWvz3l9z2b/ffbigvPPZdWq\nVfzww3TGjf2EQw/aj6OO6MzECV+kO9SkS+LNtYyREYlX0ixJR8St6y1pTEz5GkkrJf0m6RlJ1aOy\nkZLOTUfcJZGbm8uyZct45+Mx3HjLnfTtfRpmxsaNuXz26TgeffpZ3npvJMOHDmH0yI/THW7atW1U\ng7UbNjJj4SoAVqzN5a53p3P7/+3GgDM7MG/ZWnL/sDRHWXo25uby1aQv6dvvfMZ//iVVq1bjvnvu\nDH+Pli5l5Cfjue2Ou+l12imYZdfnUK6GhcxAx5lZdcIsx50IXUPKrEaNmnDM8T2QxF4dO5GjHBYv\nXkTDRo054MCDqVu3HlWrVuWIrt34erKPvNl1twa8N23L9stPflhMn0Ffcs6zk5i9ZDW/FNFUUZY1\natyExk2a0GmfMDjg/514Ml9NmkTjxk04vseJSKJjp33Iyclh0aJFaY42eUTRzQxZ19QgabfCFjb3\n500ZM5sDvAO0S/W5k6n7scczZtQIAGb8MJ0NG9ZTt249DuvSlW+mfc3q1avJzc1l3NhPaLVrmyKO\nlt0EdGnTgA++2TLx1q4abjrW2KYiJ+/dmCFfzUtDdKmxww470KRJU6Z/H+5zjxzxEa3btOG4409g\nVPSL6Ifp01m/YT316tVLZ6jJlUBttwzm3SJ7NUwlPCRREBVRnnSSmgJHE8bA3Np9+wH9AJo0bZbk\nyAp2Xp8zGDdmNEsWL6JD6x256tobObVXby69oC+H7NuBypUr8+DjTyOJWrVrc/7fL6Fb5/1B4oiu\n3Tiy29EpizUVbu3Rhr2b16LWtpUYdtH+DBg9k9/X5nJl112oXbUS//nr7kz/bSUXvzIFgD2b1WLB\ninXMWbZlt7oruu7MLg1C17Knxszm5yyu8QLc+58HObv3Gaxfv54dd9yJx58cSLVq1Ti/3zl03HN3\nKleuzICnBpXJ7lWFybbrAVBh7UGSdi2wMEZsd7NiBSHNAuoRHkfOUxn40swOiitfDrwNXGFmaySN\nBF4ws6e25pwd9trb3h/1aUnCzlrHPDg23SFkrFFXHZruEDLWQft34suJE5KaJbffuZ2dcu9/i9zu\nof9rM7EEo5OlXKE13pIm1K3UI3oCBAg31wijuudb7pwrH8pgp4UiJTwer3POpYMn3syVN+pQno1m\ntiFt0TjnkiLcPMu+zFuWupMV5jHCSEN5yzPpDcc5lywpGI835TKixmtmLfJZN4hozMv8ymO261w6\nUTnn0k2QN6daVtmqGq+k6pLaR4PjOOdcqctJYClrEopZUjVJzwG/AxOJRiST9LCk60oxPudcOZes\nBygktZA0XNJSSfOj/FUxKusgaaKk1dGfHWL2k6S7JC2OlrtVwobnRL8s7iAM9HsAmyd8A3gf+EtJ\nAnDOuYIogceFt+KR4UcJY4g3BDoAhwIXSKoMDCFMb1YbeBYYEq2H8NBVD8LcansAxwLnleS6Ek28\nJwAXmdmnbPmk2jeE2Yadc65UJPGR4R2B18xsrZnNJ8ym0xboTLjfdb+ZrTOzBwnNy4dH+50F3Gdm\nv0ZDFtwH9C7JNSWaeOsTviniVSvJyZ1zrjACKuaoyAWoJ2lCzNIvn8M9APSUVFVSY6A7m5PvFNvy\nMd4p0XqiPyfHlE2OKSuWRBO8DAF/AAAb7klEQVTvRML4CHnyAjwbGF+SAJxzrjAJ1ngXmVnHmGVA\nPocaRUiYvwO/AhOAwUB1wlAEsZYDeXNQxZcvB6qXpJ030e5k1wHDo6mNKwJ/l5RXRfeH151zpSNJ\n/XQl5QDvAU8Q7lVVBwYCdxFmTq8Zt0tNYEX0emVceU1gpZVg4OOEarxmNpqQYBsAc4ATCVO+H2hm\nPsuwc67UKIH/ElCH0Bvr4agddzHhQaujgWnAHnE12D2i9UR/to8pax9TViwJP0BhZhOBU0pyMuec\n2xrJmt7dzBZJmgn8TdK9hBrvWYT22pHARuBiSY8DfaPd8qZ+eQ64XNJwQjPrFcBDJYkn0X68VQtb\nShKAc84VpkKOilwSdCLQDVgI/EgYZvYyM1tP6C52JrCMcO+qR7QeQvPEUOBrwhjlb0frii3RGu9K\nCh/wPLumwXXOZYRk1XgBzOwrwn2p/MomAXsXUGbA1dGSFIkm3u5x7ysBexLGy70hWcE459wWyujU\nPkVJKPGa2Xv5rB4maTpwBqENxDnnkq4sTmZZlJKOLzGBzU93OOdcUuU1NfiwkJHoOea/E7qXOedc\nKRAVsrDGm1DilbSQLW+uCagFrCfcCXTOuaQT5biNF7g+7v0fhC4Z48wsvzEcnHOu5MpoU0JRiky8\n0XiVG4Dh0Yg+zjmXMuXy5pqZ5QIPA1VKPxznnNssr6khScNCZoxEmxo+JzyfPLsUY3HOuT/JxjnX\nEk28DwP3SWpEGCJyVWyhmX2T7MCcc06UzTnVipJo4n0t+vPR6M+8Hg6KXvsjw8655FOY/ifbJJp4\n25RqFM45V4DsS7tFJF5JA4FLzOz7FMXjnHObhCfXsi/1FtV8chawbSoCcc65/JTHR4bL4CU557KH\nym0bb7HnFXLOuZLI1l4NiVzTfEkbC1tKPUrnXLklqchlK47VU9K3klZJmiHp4Gh9F0nfSVotaYSk\n5jH7VJE0UNLvkuZLuryk15RIjbcfYToM55xLuWQ1NEg6kjCr8CmEh8IaRuvrAW8QJnYYCtwCvArs\nF+3aH9gFaA7sAIyQ9I2ZvVvcWBJJvEOzdSCcbHwiJhk++UfndIeQsdpc9Xa6Q8hYc39dnvRjSiRz\nWMibgX+Z2afR+znhHOoHTDOz16P3/YFFklqb2XeEERj7mNlSYKmkJ4HeQLETb1FNDd6+65xLqwSb\nGupJmhCz9Is7RgWgI1Bf0o+SfpX0sKRtgbaE2YYBMLNVwAygraTaQKPY8uh125Jck/dqcM5ltAST\n0CIz61hI+faEuSJPBg4mjLg4hDDkbXXCMLexlgM1orK89/FlxVZojdfMcrK1mcE5VzYkaXSyNdGf\nD5nZPDNbBPwbOJowi3rNuO1rAiuiMuLK88qKLRt7ajjnskToTqYil6JE7bO/kn/z6TTC6IvhnFI1\noCWh3XcpMC+2PHo9rfhX5YnXOZfRRI6KXhL0DHCRpAZR2+2lwDDgTaCdpJMkbQPcCEyJbqxBmEX9\nekm1JbUG+gKDSnJVnnidcxktiQOh3wJ8AUwHvgUmAbeZ2ULgJOA2YCmwL9AzZr+bCDfbZgOjgHtK\n0pUMSjDLsHPOlba8poZkMLMNwAXREl/2IdC6gP3WAWdHS1J44nXOZa4yOrVPUTzxOucymide55xL\nIZHUJ9cyhide51xGUxY+x+WJ1zmX0bKwwuuJ1zmX2bzG65xzKRTmXEt3FMnnidc5l7m27sm0MsMT\nr3Muo2Vf2vXE65zLYNk6vbsnXudcRsu+tOuJ1zmX6bIw83ridc5lNG9qcM65FMu+tOuJ1zmX6bIw\n8/pA6M65jCXCk2tF/bdVx5R2kbRW0gsx606TNFvSKkmDJdWJKasj6c2obLak00p6XZ54nXOZK4HZ\nJ4rRBPwIYSaKcAqpLfAE0IswG/Fq4NG47ddHZacDj0X7FJs3NTjnMloy761J6gksA8YBO0erTweG\nmtnoaJsbgG8l1QD+IEwL1M7MVgJjJL1FSNLXFDcOr/E65zJYIg0NAqgnaULM0u9PR5JqAv8Crogr\nagtMzntjZjMINdxW0bLRzKbHbD852qfYvMbrnMtoCdZ4F5lZxyK2uQV42sx+0ZYHrQ4sj9t2OVAD\n2FhIWbF54k2Bi/92Lu+/M5x69Rsw5ouvALjjXzfxzttvkZOTQ736DXjoiadp2LARAGNGj+L6f1zO\nhg251Klbl6HvfZzO8FNq151bUKN6DSpUqEDFihUZ+9kElixZQq/TTmH27Fk0b96CF15+jdq1a6c7\n1FJxV889OHy3BixeuZ5ud48G4PLurTiy3fb8Ycbileu58qXJLPh9HTs1qMY9p7anbZOa3Pf2dJ4c\n+dOm43xyw2GsXJvLH2bk/mGc8O+x6bqkEhHJ6dQgqQNwBLBnPsUrgZpx62oCKwhNDQWVFZs3NaRA\nz9PP4tXBw7ZYd+GlVzD6s0mMHD+Rrt2O5t47bgVg+bJlXH3ZRbzw2puMnTCZgc+/ko6Q0+rdD0fw\n2cSvGPvZBADuvftOOh/ehanf/kDnw7tw7913pjnC0vO/z3+l94DPt1g34OOf6H7PJxxz7xg+nraA\ni4/aBYDlqzdw8xvTeGrEzHyPddqjn3LMvWPKbNLdRAksResMtAB+ljQfuBI4SdKXwDSg/abTSTsB\nVQjTwE8HKkraJeZY7aN9is0TbwoccNDB1K5dZ4t1NWpu/hJdvXo1eT99/vfayxx7fA+aNG0GQP0G\nDVIXaIYaNnQIZ/Q6C4Azep3F0LcGpzmi0vP5T0tYtmrDFutWrsvd9HrbyhUwC68Xr1zPlF+Ws2Hj\nH6kMMeVyoqEhC1sSMABoCXSIlseBt4GjgBeB4yQdLKkaoR34DTNbYWargDeAf0mqJulA4ATg+ZJc\nkzc1pNFt/W/g1ZdfoGbN7Rg8/AMAZvz4Axs2bOD4bl1YuXIF511wEaec1ivNkaaOJI7r3hVJnNP3\nPM7p248Fv/1Gw4YNAWjYsCELFyxIc5Spd+XRu/J/HRuzYm0upz3yaZHbm8Fz5++LGbw8fjYvj/8l\nBVGWjmQ0NZjZakI3sXBMaSWw1swWAgslnU9IwHWBD4E+MbtfAAwEFgCLgb+ZWdmo8UqaJWm9pHpx\n67+SZJL2l/S7pAoxZU8WsO7x6HVbSe9LWippmaSJko5O1TWV1HX9b2HK9zM5+ZRTeeqJ0G0wNzeX\nyV99ycv/e4vXBw/n3rtu58cfphdxpOzx8aixjP/iSwYPe4cnHnuEMZ+MTndIGeHe4d9z4L8+ZsjE\nOZx5cPMitz/5wXEcd98Y+gz4nF4HtmCfneoUuU9GSqSZoRiZ2cz6m9kZMe9fMrNmZlbNzE4wsyUx\nZUvMrEdU1szMXirRNZH6poaZwKl5byTtDmwbvV0EVAD2itn+YGBu3LpDgLx/jUOBDwgdmxsAFwO/\nl0bgpemkv/Zk2JA3AWjUuAmHH3EU1apVo269ehxw4EFM+3pKmiNMnUaNwg3GBg0acHyP/+OLLz6n\nwfbbM2/ePADmzZtXrptf3vpyLt32aFjkdgt+XweE5oj3vp5P+2a1Sju0UpPsJ9cyQaoT7/PAmTHv\nzwKei15vAD4lJFYkNQAqA6/GrWsFjI5qzjsCT5rZ+mgZa2ZjUnIlJTTjxx82vX737aHs0mpXALof\ncxyfjhtDbm4uq1evZuIXX9Bq19bpCjOlVq1axYoVKza9/vCD92nbth3HHHs8Lzz/LAAvPP8sxx53\nQjrDTLkW9apuen1Eu+35acHKQrfftnIFqlWpsOn1wbvW5/v5JboJnzaiVJ5cS7tUt/F+CvSS1IZw\nt/AU4CDg1qh8NCHJ3hf9OSZaLoxZN9PMflW4G/Uj8IKkp4DxZvZbYSePOlX3AzbdvEqFvr3PYOwn\no1iyeBG7t2rBP667kQ/fe5cff5hOTo5o0qw59z3wCACtWrfh8COP4pB99yInJ4czevehTdt2KYs1\nnRb89hunnPx/AORuzOWUnqfR9ahu7N2xE2ec+leefeZpmjZtxouvvJ7mSEvPA706sN/OdaldrTLj\nbjqc+9/9gc5t6rNTg+qYGXOWruG6178GoF6NKrx1+YFU36YiZtDn0BZ0vXM0tatX4ok+oUtrhQri\nrYlzGf3dwnReVomUxcRaFFneLdLSPpE0CzgX2A+oBowiPEHSnVDb3ZHQ3eN/QD3gfmAq8DIwA9gh\nWlfTzPpEx2xCeGyve7T/GOAcM9tcnSxAh732to8++Sxp15dNqlXxe64FaXPV2+kOIWPNfelS1v32\nQ1LTZLv2e9l/3y36R2ybRtUmJvAARcZIR3ey54HTgN5sbmbI8ynhKZJ2hNrtJ9Hz0b/ErNt0t8XM\nfjWzC82sJdAcWJXPMZ1zZVg2NjWkPPGa2WzCTbajCf3jYsvWEkYNOhZoaGbfRUWfROv2ICbxxu37\nC2EUofLxu9y5cqIUOjWkXboeoDgHODzqnBxvNHApYfSgPGOidfOjASyQVFvSzZJ2lpQT3Ww7m1Br\nds5liyzMvGlJvGY2w8wmFFA8itA1LLZhZ0y0Lra2u57QJvwhoQvZVGAdoQnDOZcFpKQ9uZZRUnYX\nxcxaFLA+l5jvLDN7j7jvsKi3Qvy6VYTuaM65LFb20mrR/Pa1cy6zZWHm9cTrnMtgZfPJtKJ44nXO\nZbQy2IRbJE+8zrmMlffIcLbxxOucy2je1OCccymWjTVen4HCOZfRkvH8hKQqkp6WNFvSCkmTJHWP\nKe8i6TtJqyWNkNQ8bt+B0djg8yVdXtJr8sTrnMtcCrOSFLUkoCJhzJdDge2AG4DXJLWInnp9I1pX\nB5hAGI42T39gF8J4MIcBV0vqVpLL8qYG51zGStbNteiBq/4xq4ZJmgnsTZjuZ5qZvQ4gqT+wSFLr\naLyYM4E+ZrYUWCrpScITsu8WNx6v8TrnMlppDNUgaXvCpArTgLbA5LyyKEnPANpKqg00ii2PXrct\nxmk38Rqvcy6jJVjjrScpdvyXAWY2IP/jqRJhYstnzew7SdWB+JHilwM1CMPU5r2PLys2T7zOuYyW\nYHeyRYkMhC4phzAm+HrCzDYAK4GacZvWBFZEZXnv18aVFZs3NTjnMlqyBkKPpgt7mjA57klmtiEq\nmga0j9muGtCS0O67FJgXWx69LhvTuzvn3NZKJOluxc23x4A2wHFmtiZm/ZtAO0knSdoGuBGYEjMR\nw3PA9dEY4K2BvsCgklyXJ17nXEZLxvTuUb/c84AOwHxJK6PldDNbCJwE3AYsBfYFesbsfhPhZtts\nwnjh95hZsXs0gLfxOucyXXK6k80u7Ehm9iHQuoCydYTZbc4ueSSBJ17nXEbLwieGPfE65zJZ2Zza\npyieeJ1zGStbh4X0m2vOOZdiXuN1zmW0bKzxeuJ1zmU0HwjdOedSSIKc7Mu7nnidcxnOE69zzqWW\nNzU451yK+c0155xLsSzMu554nXOZLcE51coUT7zOuYyVrU+uyczSHUNaSFpIGOYtE9QDFqU7iAzl\nn03hMunzaW5m9ZN5QEnvEq6xKIvMrEQz/6ZSuU28mUTShESmLSmP/LMpnH8+ZZOP1eCccynmidc5\n51LME29myHcaagf4Z1MU/3zKIG/jdc65FPMar3POpZgnXuecSzFPvM45l2KeeDOQsvEZSefcJp54\nM5D5HU9XTJKOk9Q63XG4wvlYDRlEUl9gB+AzYIKZLUlzSK4MkfQqUB+4TdJMM1uX7phc/rw7WYaQ\nNBhoDKwEGgAPmNkASfIasCtKlHQbAccAa8xsQ1x5jpn9kZbg3J94U0MGkDQU2NbMOpnZYcAo4DxJ\nFct70pV0sqS26Y4jk0nqCDQDupnZ72a2QVJ9SYdKOhLAzP6Q5P/eM4Q3NaSZpMMItZTOMauHEGq/\n1YDlaQgrI0hqB9wCvCXpaTObnu6YMpQBNcxslaQGwF7Ac8BSoLak0WZ2std4M4d/A6aZmY0AbgLe\nk9RBUiXgSeBjMyu3SRfAzKYCZwL7AedIapVXJqlCzOvt0hBe2km6TNLZZjYR2EbSFOAZ4D/AQ8Ch\nwGnAPnk1X5cZvMabRnntbmZ2S/QzcAKwBrjWzB6Ktil3bbySbgdaAguB+4FLCInEJD1jZt+b2cZo\n20uA9pLON7P1aQs6xSRVA/YlVJ4GEmq5twKTgZlm9nG03WrCuNMr0hSqy4ffXEsDSXsD081sRdz6\ni4AHgHZm9o2kSvE3SbKdpCGEga8/AjoQEksPQu3tJmA8MNDMvpd0JXA3sLeZTUpTyGkjqSUwEbjH\nzG4rYJvzgEuBI8xsTirjcwXzGm+KSboVuBYYI+k2YIqZzQMws4ck1Qe+knSUmY0oT3ejJf0XaJw3\nsLekHsCdQEMz+0iSAf2BZVFb5vlAp/KUdCWdDzwF5JjZDEl/A/pLGp9Xy422Owg4ELgK6OpJN7N4\n4k29qcBMYAkwCJgk6WPgYWCdmd0oaSPwkaRDzGxM+kJNHUl1gKOAByRVNbPVwDfAaqB61OTycfRQ\n34OEm48HRe2b5YKkE4FHCW3eH0oaDbxO+Ny6S5poZsujZoju0XaHmdnXaQva5cubGlJE0jZmtlbS\nTsAI4ErgS+AMoDehPfMdws/GlZIuBD4ys2/TFXOqSLqR0ITwK/AW4QvpHWAw8LCZ3R23fUfCHFuz\nUhtpeklqSvhMVhESbndCjbYJ4Yu7t5mNjbbdltBF0R/CyUDeqyEFJD0JnA5gZj8B9xLaJv8ws5sJ\nSXhv4GRgnqTLgcfKSdKtDWwL3AbUAU4AzgbGAS/mJd3YPqhmNqG8JV0AM/sFuAioAPwEvAYMBSoD\nc4FHJNWItl3jSTdzeeItZdHNoj3N7OmY1SOA34Bmkg4hdP+5ANid0P47OO+ufbYzs6XAI4S+y/8G\nagNdgTmEttw60Xblop07nqQbJI2XtHPUBDMOGAbsZmaDgAuBIwgzDe9B+HvkMpw3NZQiSW8C9c3s\noJh1lc1svaRHgV6ER4SvM7OB6YozHaKn8nJj3lcCbgeOBfoAvxNqcy8THp9emJZA00jSPwhPMd4G\nVAI+BB4jjOdxJ3CZmU2X1AzYhdCdrI+ZfZemkF2CPPGWEklPEW56tIjpc7oT4cbQhcA64ANgtJld\nUJ7660q6g3CT8b/AejMzSY0Ibd5TgRrAFYRa3HhCTfi28lTrjdr4HwTeB/oBxwFdgDZAT8KDJQcA\nh+b1Xy5PPWDKOm9qKD3PE+7IdwWQ1AQYDXwVtU8uAyYRai/lbSjIEcCNwHFR0m1KGJHtAcKX1WBC\nG3hDwkMCL5fDhPI28AmwD6Fp4W1C97m3Ccl4CaFZ5vy88ZvL4WdUZnniLSVmNopQU/l31PdyPPCo\nmV0f1W7XEJLNYZJqlafBz83sfaAvcEv02YwDHjezO6JfBy8BHwPXA7+Y2Y/pizb1oseh5xKS7FmE\nL+jRQDUzu5Lwi6kN0Ao4EdgmTaG6YvKmhlIW3TwbDgwzs54x6y8j/FzsaWbfpyu+dMrvs8n7uRw1\nPawzs8VpDTKFJO0PfB7TNNUHuIxQ632CcBPtIDObKakxoZ/u1PL696cs88SbApIOJDxtdJ2ZvRHV\n8u4gPMZZbh4AyE/02TwJXAe8Fz04Ue5IGkT4In6K0OPlpugL6E5gnpk9oDBm8x6EJ9HK1a+AbOOJ\nN0UkHUoY6OUrwjCQR5rZl+mNKjNEn82jhLvy/ytPg93kkXQAoe32OWBHQMB7hHbuymZ2abTdB4T7\nAnvG9gpxZYsn3hSKxt59iTBg9eR0x5NJJB1B+BVwePzgQeVFVPt/HLicMLB5DUI7dx3gKDP7INqu\niZn9mrZAXYl54k2xvEeH0x1HJooZo6Hcir6cHwRuNLM3JXUAdiY8Qr3Gey5kB0+8zmUYSZ0J7d7/\nNLP/pjkcVwp8dDLnMoyZjZR0LvBY9ERfuWz3zmZe43UuQ3m7d/byxOtcBvN27+zkidc551LMHxl2\nzrkU88TrnHMp5onXOedSzBOvc86lmCde55xLMU+8rtgkTZXUP+b9LElXpiGOjpJMUotSPEd/SVOT\ncJy0fEYus3jizSKSBkUJyCRtkPSTpHslVUtRCJ0Io4wVSVJvSStLOZ7Y842U9HCqzudcYfyR4ezz\nIWESzUrAwYTxXasBf8tvY0mVzGxDMk5cHiekdK44vMabfdaZ2Xwz+8XMXgJeBHpAGHwlqg0fLelz\nSesJc5wh6ThJEyWtlTRT0m2SKucdVFIDSUMkrZE0W9LZ8SeO/xktqaakxyTNi477raRTokFgngGq\nxdTQ+0f7VJZ0l6RfJa2S9IWko+LO003Sd9ExPyFMgVMiku6U9H10fbMk3S3pT1PqSDpX0s/RdoMl\n1Ysr7yPpmyi26ZIuk+T/ztwWvMab/dYQar+x7iLM4vsjsCJKbC8ClxDm9mpGGBe2CpCXSAcBzQnT\nz6wG/gO0KOik0Rxy7xAmZOwDTAd2JcwPNg64lDCde8tol7xmh2eidacBvwJHA0MldTKzydHEmIMJ\no3c9QpiR4d+JfxwFWgWcDcwBdiNc/zrghphtWgBnACcAVYEBwEDg+Oia+wL/Ai4CJgLtojg3AN7M\n4TYzM1+yZCEkx2Ex7/chTJH+avS+M2DASXH7jQZuiFvXg5AMRahRGnBgTHlzYCPQP2bdLODK6PWR\nwB9AmwJi7Q2sjFvXMtqnWdz6wYSJQiEk6+lEj7tH666P4mtRyGczEnh4Kz7L84EfY973j663Wcy6\ng6Lz7hK9/xnoFXecS4Fv8vuMfCm/i9d4s0+36KZVRUJNdwihBhZrQtz7vYF9JP0jZl0OsC1hmpk2\nhIT4eV6hmc2WNLeQOPYkzBX27VbEvhch0X8TN+lyFcKsw0SxfGpmsYOMjN+Kc+RL0smEJLkzUB2o\nEC2x5pjZzzHvPyP6cpG0DGgKPCHpsZhtKhKuyblNPPFmn9GEaeU3AHMt/xtnq+Le5wA3A6/ns+1C\nipc4irNPDqEG2YkQf6w1JThuoSTtB7xC+AwuA5YRmg/u3YrD5LXj5k1X71yBPPFmn9W29TPQfgm0\nLmg/Sd8SEksnoqQiqRnQqIhjNpTUpoBa73r+XKOcREisO5jZiAKO+w1wkiTF1Hr3KySORBxIqM3e\nkrdCUvN8tmssqamZ/RK934fwuXxrZr9JmgO0NLPnShiPy3KeeB2EG0LDJM0GXgNyCTeG9jGzq83s\ne0nvEn5G9yPUPv/N5lpofj4i/BT/n6TLCO2yOwPVzGwwoa1zG0lHEhLuajObLulFYJCkKwjJuw6h\nbfonM3uDcNPrCuB+SY8CuxNqmYmoF81hFmtBFFtjSacTmi2OAk7NZ/81wLOSLic0wzwOvG1mP0Tl\n/YGHomaH4YSmnr2AxmZ2R4IxunLAu7k4zOw9wpTzhxHacT8HriHcLMrTG5hJaGsdSpgteVYhx/wD\n6A6MBV4AvgUeACpH5eMIietlQnPG1dGufQg9G+4GvgOGAYcAs6P9fgZOBLoBkwlNA9ckeKmnEJJ8\n7HK5mQ0F7gHuB6YQbgzemM/+swhNEkOjz+GnKN68a36K0DOiVxTbJ4Rmn5kJxufKCR8I3TnnUsxr\nvM45l2KeeJ1zLsU88TrnXIp54nXOuRTzxOuccynmidc551LME69zzqWYJ17nnEux/wcUEWKdBzv5\npAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2a387b82da0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Confusion Matrix\n",
    "conf_mx = confusion_matrix(y_test, y_pred)\n",
    "plot_conf_matrix(conf_mx, train.author.unique(), title=\"SVC Pipe Confusion Matrix\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### GridSearch Parameters\n",
    "    MNB\n",
    "    clf__alpha: 0.01\n",
    "    clf__alpha: 0.01\n",
    "    tfidf__norm: 'l2'\n",
    "    tfidf__use_idf: False\n",
    "    vect__max_df: 0.5\n",
    "    vect__max_features: None\n",
    "    vect__ngram_range: (1, 2)\n",
    "\n",
    "    SGD\n",
    "    clf__alpha: 1e-05\n",
    "    clf__n_iter: 50\n",
    "    clf__penalty: 'l2'\n",
    "    tfidf__norm: 'l2'\n",
    "    tfidf__use_idf: True\n",
    "    vect__max_df: 1.0\n",
    "    vect__max_features: None\n",
    "    vect__ngram_range: (1, 2)\n",
    "\n",
    "    SVC\n",
    "    clf__C: 10\n",
    "    clf__tol: 0.01\n",
    "    tfidf__norm: 'l2'\n",
    "    tfidf__use_idf: True\n",
    "    vect__max_df: 0.75\n",
    "    vect__max_features: None\n",
    "    vect__ngram_range: (1, 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Try different classifiers\n",
    "http://scikit-learn.org/stable/auto_examples/text/document_classification_20newsgroups.html#sphx-glr-auto-examples-text-document-classification-20newsgroups-py\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 412,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(14684,) (4895,) (14684,) (4895,)\n"
     ]
    }
   ],
   "source": [
    "from time import time\n",
    "\n",
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "from sklearn.feature_selection import SelectFromModel\n",
    "from sklearn.feature_selection import SelectKBest, chi2\n",
    "from sklearn.svm import LinearSVC\n",
    "from sklearn.svm import SVC\n",
    "from sklearn.linear_model import RidgeClassifier\n",
    "from sklearn.linear_model import SGDClassifier\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.naive_bayes import BernoulliNB\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.neighbors import NearestCentroid\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn import metrics\n",
    "from sklearn.utils.extmath import density\n",
    "\n",
    "target_names = train.author.unique()\n",
    "X_train, X_test, y_train, y_test = train_test_split(train['text'], train['author'])\n",
    "print(X_train.shape, X_test.shape, y_train.shape, y_test.shape)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 413,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting features from training data using a sparse vectorizer\n",
      "0.4792783260345459\n",
      "(14684, 22389)\n",
      "(4895, 22389)\n"
     ]
    }
   ],
   "source": [
    "print(\"Extracting features from training data using a sparse vectorizer\")\n",
    "t0 = time()\n",
    "vectorizer = TfidfVectorizer(sublinear_tf=True)\n",
    "X_train = vectorizer.fit_transform(X_train)\n",
    "X_test = vectorizer.transform(X_test)\n",
    "duration = time() - t0\n",
    "print(duration)\n",
    "print(X_train.shape)\n",
    "print(X_test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 414,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['ab', 'aback', 'abaft', ..., 'élite', 'οἶδα', 'υπνος'],\n",
       "      dtype='<U19')"
      ]
     },
     "execution_count": 414,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "feature_names = vectorizer.get_feature_names()\n",
    "if feature_names:\n",
    "    feature_names = np.asarray(feature_names)\n",
    "\n",
    "feature_names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 415,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "None\n"
     ]
    }
   ],
   "source": [
    "print(vectorizer.get_stop_words())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 416,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Benchmark classifiers\n",
    "def benchmark(clf):\n",
    "    print('_' * 80)\n",
    "    print(\"Training \")\n",
    "    print(clf)\n",
    "    t0 = time()\n",
    "    clf.fit(X_train, y_train)\n",
    "    train_time = time() - t0\n",
    "    print(\"Training time: %0.3fs\" % train_time)\n",
    "    \n",
    "    t0 = time()\n",
    "    pred = clf.predict(X_test)\n",
    "    #pred_prob = clf.predict_proba(X_test)\n",
    "    \n",
    "    \n",
    "    score = accuracy_score(y_test, pred)\n",
    "    \n",
    "    \n",
    "    \n",
    "    \n",
    "    if hasattr(clf, \"predict_proba\"):\n",
    "        pred_prob = clf.predict_proba(X_test)\n",
    "    else:\n",
    "        pred_prob = clf.decision_function(X_test)\n",
    "        pred_prob = (pred_prob - pred_prob.min()) / (pred_prob.max() - pred_prob.min())\n",
    "    \n",
    "    \n",
    "    lnloss = log_loss(y_test, pred_prob)\n",
    "    test_time = time() - t0\n",
    "    print(\"Testing time: %0.3fs\" % test_time)\n",
    "    print(\"Accuracy score: %0.3f\" % score)\n",
    "    print(\"Log loss: %0.3f\" % lnloss)\n",
    "    \n",
    "    \n",
    "    if hasattr(clf, 'coef_'):\n",
    "        print(\"Dimensionality: %d\" % clf.coef_.shape[1])\n",
    "        print(\"Density: %f\" % density(clf.coef_))\n",
    "        \n",
    "        if feature_names is not None:\n",
    "            print(\"Top 10 keywords per class: \")\n",
    "            for i, label in enumerate(target_names):\n",
    "                top10 = np.argsort(clf.coef_[i])[-10:]\n",
    "                print(\"%s: %s\" % (label, \" \".join(feature_names[top10])))\n",
    "            print()\n",
    "    print(\"Classification Report:\")\n",
    "    print(classification_report(y_test, pred, target_names=target_names))\n",
    "    \n",
    "    print(\"Confusion Matrix:\")\n",
    "    print(confusion_matrix(y_test, pred))\n",
    "    \n",
    "    print()\n",
    "    clf_desc = str(clf).split('(')[0]\n",
    "    return clf_desc, score, train_time, test_time\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 417,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "================================================================================\n",
      "LR\n",
      "________________________________________________________________________________\n",
      "Training \n",
      "LogisticRegression(C=0.1, class_weight=None, dual=False, fit_intercept=True,\n",
      "          intercept_scaling=1, max_iter=100, multi_class='multinomial',\n",
      "          n_jobs=1, penalty='l2', random_state=None, solver='lbfgs',\n",
      "          tol=0.0001, verbose=0, warm_start=False)\n",
      "Training time: 0.654s\n",
      "Testing time: 0.015s\n",
      "Accuracy score: 0.743\n",
      "Log loss: 0.803\n",
      "Dimensionality: 22389\n",
      "Density: 1.000000\n",
      "Top 10 keywords per class: \n",
      "EAP: say no has very it at have however is upon\n",
      "HPL: men they street things west seemed and had old though\n",
      "MWS: on perdita our love you me she raymond my her\n",
      "\n",
      "Classification Report:\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        EAP       0.68      0.87      0.76      1949\n",
      "        HPL       0.81      0.66      0.73      1423\n",
      "        MWS       0.81      0.66      0.73      1523\n",
      "\n",
      "avg / total       0.76      0.74      0.74      4895\n",
      "\n",
      "Confusion Matrix:\n",
      "[[1696  104  149]\n",
      " [ 402  934   87]\n",
      " [ 405  113 1005]]\n",
      "\n",
      "================================================================================\n",
      "L2 penalty\n",
      "________________________________________________________________________________\n",
      "Training \n",
      "LinearSVC(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
      "     intercept_scaling=1, loss='squared_hinge', max_iter=1000,\n",
      "     multi_class='ovr', penalty='l2', random_state=None, tol=0.001,\n",
      "     verbose=0)\n",
      "Training time: 0.335s\n",
      "Testing time: 0.020s\n",
      "Accuracy score: 0.831\n",
      "Log loss: 0.843\n",
      "Dimensionality: 22389\n",
      "Density: 1.000000\n",
      "Top 10 keywords per class: \n",
      "EAP: evident however altogether rigid precisely minutes madame lady dupin upon\n",
      "HPL: jermyn innsmouth whilst amongst gilman west birch later despite though\n",
      "MWS: cottage quitted elizabeth plague misery idris towards perdita adrian raymond\n",
      "\n",
      "Classification Report:\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        EAP       0.82      0.85      0.83      1949\n",
      "        HPL       0.84      0.82      0.83      1423\n",
      "        MWS       0.84      0.82      0.83      1523\n",
      "\n",
      "avg / total       0.83      0.83      0.83      4895\n",
      "\n",
      "Confusion Matrix:\n",
      "[[1656  137  156]\n",
      " [ 167 1172   84]\n",
      " [ 202   79 1242]]\n",
      "\n",
      "________________________________________________________________________________\n",
      "Training \n",
      "SGDClassifier(alpha=0.0001, average=False, class_weight=None, epsilon=0.1,\n",
      "       eta0=0.0, fit_intercept=True, l1_ratio=0.15,\n",
      "       learning_rate='optimal', loss='hinge', max_iter=None, n_iter=50,\n",
      "       n_jobs=1, penalty='l2', power_t=0.5, random_state=None,\n",
      "       shuffle=True, tol=None, verbose=0, warm_start=False)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\anoop\\Anaconda3\\lib\\site-packages\\sklearn\\linear_model\\stochastic_gradient.py:117: DeprecationWarning: n_iter parameter is deprecated in 0.19 and will be removed in 0.21. Use max_iter and tol instead.\n",
      "  DeprecationWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training time: 0.333s\n",
      "Testing time: 0.015s\n",
      "Accuracy score: 0.830\n",
      "Log loss: 0.840\n",
      "Dimensionality: 22389\n",
      "Density: 0.827683\n",
      "Top 10 keywords per class: \n",
      "EAP: immediately altogether is balloon minutes dupin madame however lady upon\n",
      "HPL: uncle old whilst street innsmouth gilman despite later west though\n",
      "MWS: quitted misery love plague idris her perdita towards adrian raymond\n",
      "\n",
      "Classification Report:\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        EAP       0.81      0.85      0.83      1949\n",
      "        HPL       0.85      0.82      0.83      1423\n",
      "        MWS       0.85      0.81      0.83      1523\n",
      "\n",
      "avg / total       0.83      0.83      0.83      4895\n",
      "\n",
      "Confusion Matrix:\n",
      "[[1660  133  156]\n",
      " [ 183 1171   69]\n",
      " [ 212   81 1230]]\n",
      "\n",
      "================================================================================\n",
      "L1 penalty\n",
      "________________________________________________________________________________\n",
      "Training \n",
      "LinearSVC(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
      "     intercept_scaling=1, loss='squared_hinge', max_iter=1000,\n",
      "     multi_class='ovr', penalty='l1', random_state=None, tol=0.001,\n",
      "     verbose=0)\n",
      "Training time: 0.963s\n",
      "Testing time: 0.014s\n",
      "Accuracy score: 0.816\n",
      "Log loss: 0.914\n",
      "Dimensionality: 22389\n",
      "Density: 0.143612\n",
      "Top 10 keywords per class: \n",
      "EAP: evident balloon lady treatise dames rigid upon prodigies madame dupin\n",
      "HPL: birch warren slater amongst musides later johansen gilman whilst despite\n",
      "MWS: towards labour ryland labours windsor misery idris perdita adrian raymond\n",
      "\n",
      "Classification Report:\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        EAP       0.80      0.85      0.82      1949\n",
      "        HPL       0.83      0.79      0.81      1423\n",
      "        MWS       0.83      0.79      0.81      1523\n",
      "\n",
      "avg / total       0.82      0.82      0.82      4895\n",
      "\n",
      "Confusion Matrix:\n",
      "[[1653  133  163]\n",
      " [ 201 1129   93]\n",
      " [ 220   93 1210]]\n",
      "\n",
      "________________________________________________________________________________\n",
      "Training \n",
      "SGDClassifier(alpha=0.0001, average=False, class_weight=None, epsilon=0.1,\n",
      "       eta0=0.0, fit_intercept=True, l1_ratio=0.15,\n",
      "       learning_rate='optimal', loss='hinge', max_iter=None, n_iter=50,\n",
      "       n_jobs=1, penalty='l1', power_t=0.5, random_state=None,\n",
      "       shuffle=True, tol=None, verbose=0, warm_start=False)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\anoop\\Anaconda3\\lib\\site-packages\\sklearn\\linear_model\\stochastic_gradient.py:117: DeprecationWarning: n_iter parameter is deprecated in 0.19 and will be removed in 0.21. Use max_iter and tol instead.\n",
      "  DeprecationWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training time: 0.855s\n",
      "Testing time: 0.017s\n",
      "Accuracy score: 0.782\n",
      "Log loss: 0.940\n",
      "Dimensionality: 22389\n",
      "Density: 0.049995\n",
      "Top 10 keywords per class: \n",
      "EAP: gentleman altogether evident minutes dupin however lady balloon madame upon\n",
      "HPL: uncle birch amongst innsmouth gilman west whilst later though despite\n",
      "MWS: labours love labour windsor misery towards idris perdita adrian raymond\n",
      "\n",
      "Classification Report:\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        EAP       0.75      0.83      0.79      1949\n",
      "        HPL       0.81      0.75      0.78      1423\n",
      "        MWS       0.81      0.75      0.78      1523\n",
      "\n",
      "avg / total       0.78      0.78      0.78      4895\n",
      "\n",
      "Confusion Matrix:\n",
      "[[1625  145  179]\n",
      " [ 265 1061   97]\n",
      " [ 273  109 1141]]\n",
      "\n",
      "================================================================================\n",
      "Elastic-Net penalty\n",
      "________________________________________________________________________________\n",
      "Training \n",
      "SGDClassifier(alpha=0.0001, average=False, class_weight=None, epsilon=0.1,\n",
      "       eta0=0.0, fit_intercept=True, l1_ratio=0.15,\n",
      "       learning_rate='optimal', loss='hinge', max_iter=None, n_iter=50,\n",
      "       n_jobs=1, penalty='elasticnet', power_t=0.5, random_state=None,\n",
      "       shuffle=True, tol=None, verbose=0, warm_start=False)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\anoop\\Anaconda3\\lib\\site-packages\\sklearn\\linear_model\\stochastic_gradient.py:117: DeprecationWarning: n_iter parameter is deprecated in 0.19 and will be removed in 0.21. Use max_iter and tol instead.\n",
      "  DeprecationWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training time: 0.932s\n",
      "Testing time: 0.018s\n",
      "Accuracy score: 0.823\n",
      "Log loss: 0.859\n",
      "Dimensionality: 22389\n",
      "Density: 0.462549\n",
      "Top 10 keywords per class: \n",
      "EAP: character balloon is altogether minutes dupin madame however lady upon\n",
      "HPL: birch uncle whilst gilman innsmouth street despite later west though\n",
      "MWS: miserable misery plague love idris her perdita towards adrian raymond\n",
      "\n",
      "Classification Report:\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        EAP       0.80      0.85      0.82      1949\n",
      "        HPL       0.84      0.81      0.82      1423\n",
      "        MWS       0.84      0.80      0.82      1523\n",
      "\n",
      "avg / total       0.82      0.82      0.82      4895\n",
      "\n",
      "Confusion Matrix:\n",
      "[[1652  135  162]\n",
      " [ 195 1153   75]\n",
      " [ 215   86 1222]]\n",
      "\n",
      "================================================================================\n",
      "Naive Bayes\n",
      "________________________________________________________________________________\n",
      "Training \n",
      "MultinomialNB(alpha=0.01, class_prior=None, fit_prior=True)\n",
      "Training time: 0.043s\n",
      "Testing time: 0.019s\n",
      "Accuracy score: 0.833\n",
      "Log loss: 0.426\n",
      "Dimensionality: 22389\n",
      "Density: 1.000000\n",
      "Top 10 keywords per class: \n",
      "EAP: my that was is it in and to of the\n",
      "HPL: it he that had in was to of and the\n",
      "MWS: that me her in was my to of and the\n",
      "\n",
      "Classification Report:\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        EAP       0.82      0.84      0.83      1949\n",
      "        HPL       0.86      0.83      0.85      1423\n",
      "        MWS       0.82      0.82      0.82      1523\n",
      "\n",
      "avg / total       0.83      0.83      0.83      4895\n",
      "\n",
      "Confusion Matrix:\n",
      "[[1646  120  183]\n",
      " [ 154 1178   91]\n",
      " [ 203   67 1253]]\n",
      "\n",
      "________________________________________________________________________________\n",
      "Training \n",
      "BernoulliNB(alpha=0.01, binarize=0.0, class_prior=None, fit_prior=True)\n",
      "Training time: 0.048s\n",
      "Testing time: 0.023s\n",
      "Accuracy score: 0.837\n",
      "Log loss: 0.687\n",
      "Dimensionality: 22389\n",
      "Density: 1.000000\n",
      "Top 10 keywords per class: \n",
      "EAP: at with was that it in to and of the\n",
      "HPL: it he had that was in to of and the\n",
      "MWS: me with that was my in to of and the\n",
      "\n",
      "Classification Report:\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        EAP       0.82      0.85      0.84      1949\n",
      "        HPL       0.87      0.83      0.85      1423\n",
      "        MWS       0.83      0.82      0.82      1523\n",
      "\n",
      "avg / total       0.84      0.84      0.84      4895\n",
      "\n",
      "Confusion Matrix:\n",
      "[[1660  114  175]\n",
      " [ 153 1186   84]\n",
      " [ 209   65 1249]]\n",
      "\n",
      "================================================================================\n",
      "LinearSVC with L1 based regularization\n",
      "________________________________________________________________________________\n",
      "Training \n",
      "Pipeline(memory=None,\n",
      "     steps=[('feature_selection', SelectFromModel(estimator=LinearSVC(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
      "     intercept_scaling=1, loss='squared_hinge', max_iter=1000,\n",
      "     multi_class='ovr', penalty='l1', random_state=None, tol=0.001,\n",
      "     verbose=0),\n",
      "        norm_order=1, prefit=...ax_iter=1000,\n",
      "     multi_class='ovr', penalty='l2', random_state=None, tol=0.0001,\n",
      "     verbose=0))])\n",
      "Training time: 1.190s\n",
      "Testing time: 0.022s\n",
      "Accuracy score: 0.821\n",
      "Log loss: 0.870\n",
      "Classification Report:\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        EAP       0.80      0.85      0.82      1949\n",
      "        HPL       0.83      0.81      0.82      1423\n",
      "        MWS       0.83      0.80      0.82      1523\n",
      "\n",
      "avg / total       0.82      0.82      0.82      4895\n",
      "\n",
      "Confusion Matrix:\n",
      "[[1649  136  164]\n",
      " [ 190 1148   85]\n",
      " [ 212   91 1220]]\n",
      "\n",
      "================================================================================\n",
      "________________________________________________________________________________\n",
      "Training \n",
      "Pipeline(memory=None,\n",
      "     steps=[('feature_selection', SelectFromModel(estimator=LinearSVC(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
      "     intercept_scaling=1, loss='squared_hinge', max_iter=1000,\n",
      "     multi_class='ovr', penalty='l1', random_state=None, tol=0.001,\n",
      "     verbose=0),\n",
      "        norm_order=1, prefit=False, threshold=None)), ('classification', MultinomialNB(alpha=0.01, class_prior=None, fit_prior=True))])\n",
      "Training time: 0.986s\n",
      "Testing time: 0.019s\n",
      "Accuracy score: 0.819\n",
      "Log loss: 0.499\n",
      "Classification Report:\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        EAP       0.79      0.86      0.82      1949\n",
      "        HPL       0.85      0.79      0.82      1423\n",
      "        MWS       0.83      0.80      0.81      1523\n",
      "\n",
      "avg / total       0.82      0.82      0.82      4895\n",
      "\n",
      "Confusion Matrix:\n",
      "[[1667  121  161]\n",
      " [ 202 1126   95]\n",
      " [ 236   73 1214]]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "results = []\n",
    "\n",
    "for clf, name in (\n",
    "    (LogisticRegression(multi_class=\"multinomial\", solver='lbfgs', C=0.1), \"LR\"),\n",
    "    #(RidgeClassifier(tol=1e-2, solver=\"lsqr\"), \"Ridge Classifier\"),\n",
    "    #(KNeighborsClassifier(n_neighbors=10), \"kNN\"),\n",
    "    #(RandomForestClassifier(n_estimators=100), \"Random forest\")\n",
    "):\n",
    "    print('=' * 80)\n",
    "    print(name)\n",
    "    results.append(benchmark(clf))\n",
    "    \n",
    "for penalty in ['l2', 'l1']:\n",
    "    print('=' * 80)\n",
    "    print(\"%s penalty\" % penalty.upper())\n",
    "    # LibLinear Model\n",
    "    results.append(benchmark(LinearSVC(penalty=penalty, dual=False, tol=1e-3)))\n",
    "    # SGD Model\n",
    "    results.append(benchmark(SGDClassifier(alpha=.0001, n_iter=50, penalty=penalty)))\n",
    "    \n",
    "# SGD with Elastic-Net penalty\n",
    "print('=' * 80)\n",
    "print(\"Elastic-Net penalty\")\n",
    "results.append(benchmark(SGDClassifier(alpha=.0001, n_iter=50, penalty=\"elasticnet\")))\n",
    "\n",
    "# Train Naive Bayes classifiers\n",
    "print('=' * 80)\n",
    "print(\"Naive Bayes\")\n",
    "results.append(benchmark(MultinomialNB(alpha=.01)))\n",
    "results.append(benchmark(BernoulliNB(alpha=.01)))\n",
    "\n",
    "print('=' * 80)\n",
    "print(\"LinearSVC with L1 based regularization\")\n",
    "# The smaller C, the stronger the regularization.\n",
    "# The more regularization, the more sparsity\n",
    "results.append(benchmark(Pipeline([\n",
    "    ('feature_selection', SelectFromModel(LinearSVC(penalty=\"l1\", dual=False, tol=1e-3))),\n",
    "    ('classification', LinearSVC(penalty=\"l2\"))\n",
    "])))\n",
    "\n",
    "print('=' * 80)\n",
    "results.append(benchmark(Pipeline([\n",
    "    ('feature_selection', SelectFromModel(LinearSVC(penalty=\"l1\", dual=False, tol=1e-3))),\n",
    "    ('classification', MultinomialNB(alpha=.01))\n",
    "])))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 385,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Plots of the benchmark acitivty\n",
    "indices = np.arange(len(results))\n",
    "results = [[x[i] for x in results] for i in range(4)]\n",
    "\n",
    "clf_names, score, training_time, testing_time = results\n",
    "training_time = np.array(training_time) / np.max(training_time)\n",
    "testing_time = np.array(testing_time) / np.max(testing_time)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 386,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAuMAAAI4CAYAAADAhOdIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzs3WuYnWV5//3vj4RNYgJUQCQoBtyh\nxTZ/hoIo1NBaK1Tc1Kq4eQTqBkRFQVq1KgRrLR61eKhoqSKKiBRRaHELwtM0QkGYgbBzXwGN9A+E\nCiQxoRDP58W6p13Ok2TWJBOuSfL9HMccWeu+r+u6z3t4wW+dc621UlVIkiRJeuRt1boASZIkaUtl\nGJckSZIaMYxLkiRJjRjGJUmSpEYM45IkSVIjhnFJkiSpEcO4JGlKSTI3SSWZ3uj685MsaXFtSVse\nw7gkaa2S3J5kZZLlSX6Z5OtJHt+6rqkiyTZJ/j7Jku53dFuSj7SuS9KmwzAuSRrP4VU1C9gNuAv4\neON6Npr16Ma/G9gP2B+YDRwC3NC4JkmbEMO4JGkgVbUK+DLw9NFjSbZN8uEkP0tyV5Izk8zozs3v\nOsbvSHJ3kv9McnTf3BldV/mOJPcnuXJ0bufV3bpLk7ynb96CJBcm+UKSZUluTvKUJO/urvPzJM/r\nG390ku93Y3+a5Ji+c6M1vjPJ/wU+O/a+kxyf5HtJHreGX8vvARdX1Z3Vc3tVfb5v7uOTXJTkniT3\nJjmjO75Vkvd29353ks8n2aE7N7pN53VJfgb8v93xZyb59yT3Jbkxyfy+6xzV3duyrjv/6vH/i0qa\nCgzjkqSBJJkJvAK4pu/wh4CnAPOAJwG7Ayf3nX8ssEN3/HXAJ5L8Vnfuw8AQ8Czg0cBfAr/um3sQ\n8FTgD4GTkzyt79zhwLnAb9HrRF9K7/9puwPvB/6xb+zdwAuA7YGjgY8k2XdMjY8GngC8ccw9vw84\nCnhOVa1pH/k1wIlJjkvyjCTpmzsN+BpwBzC3q+2futNHdT+HAHsBs4Azxqz9HOBpwB8n2R34OvCB\nrtaTgK8k2SXJo4CPAYdW1Wx6v8/Fa6hV0hSUqmpdgyRpikpyO7Az8DC9wHg38MdVdXMXPJcDv1NV\n/9GNPxD4YlXt2XVuvwnMrqqHu/N3Ay8ErgVWAM+sqhvHXHMucBvw+NEAnORa4PSq+qckC4BnV9Uf\ndecOB84Hdqiq1UlmAw8Av1VV963hnv4Z+Neq+mhX42XA9l3nn+7YecAF9Laf/ElV3b+W38804Fjg\nVfS2q9wLvLuqzul+F5cAu43ef9+8K4CvVNUnu+dPBW4BZgCP6+7/iVX10+78O4F9qur/6VvjUuCL\n9P5a8Qt6L3a+UVUr11SrpKnJzrgkaTwvrqodgW2BtwD/luSxwC7ATGCk2zpxH/Ct7vioe8cE0V/R\nC/U7A9sB/7GO6/7fNcwbdVff45XA0qpa3fec0fFJDk1yTZL/6mo8rLv+qHtGg3ifHel1yf92bUEc\noKpWV9UnqurZ3Zy/Ac7uuviPB+4YG8Q7c+h1zEfdAUwHdu079vO+x08AXjb6e+7u4yB6QX8Fvb9Y\nHAv8Z/cm273XVrOkqcUwLkkaSBc8LwJW0wuCS+kF39+uqh27nx26N3uOZymwCnjixqu4t6cd+Aq9\nLTG7di8qvgGkb9ia/kT8S3pbWz6b5NmDXKuqVlbVJ7q5T6cXpvdYyxsw76QXsEftQe+vD/0vMvrr\n+jlwbt/veceqelRVndZd+9LuLwW7AT8APj1IzZLaM4xLkgaSnhfR26f9/ar6Nb3Q95Ekj+nG7J7k\nj8dbq5t7NnB6kjlJpiU5sAvPk2kbeh39e4CHkxwKPG/dU/6nxoXAq4GLkxywpjFJ3t69CXRGkulJ\njqT3qSo30NuK85/AaUkelWS7vmB/PnBCkj2TzAI+CFywli46wBeAw5P8cfe72q677uOS7Jrkhd3e\n8QfpbR1avZZ1JE0xhnFJ0ni+mmQ5vX3YfwMcWVW3dufeCfwEuCbJA8Dl9N50OYiTgJuB64D/ovdm\n0En9/1JVLQOOB75Er2P9Knr7uAed/216b/q8JMnQGoasBP6e3paapcCbgZdW1U+7bTOH03tj68+A\nJfS2k0Dvhci5wCJ6+8NXAW9dRx0/B14E/BW9FxY/B/6C3u9rK+Ad9Lrt/0XvjZ/HDXqPktryDZyS\nJElSI3bGJUmSpEYM45IkSVIjhnFJkiSpEcO4JEmS1MiaPvtUmjJ23nnnmjt3busyJEmSJmRkZGRp\nVe0y3jjDuKa0uXPnMjw83LoMSZKkCUlyx/ij3KYiSZIkNWMYlyRJkhoxjEuSJEmNGMYlSZKkRgzj\nkiRJUiOGcUmSJKkRw7gkSZLUiGFckiRJasQwLkmSJDViGJckSZIaMYxLkiRJjRjGJUmSpEYM45Ik\nSVIjhnFJkiSpEcO4JEmS1IhhXJIkSWrEMC5JkiQ1YhiXJEmSGjGMS5IkSY0YxiVJkqRGDOOSJElS\nI4ZxSZIkqRHDuKa2u0ZaVyBJkrTRGMYlSZKkRgzjkiRJUiOGcUmSJKkRw7gkSZLUiGFckiRJasQw\nLkmSJDViGJckSZIaMYxLkiRJjRjGNbXtOtS6AkmSpI3GMC5JkiQ1YhiXJEmSGjGMS5IkSY1Mb12A\ntC4jy5aRhQtblyFJkgZQ8+e3LmGTY2dckiRJasQwLkmSJDViGJckSZIaMYxLkiRJjRjGJUmSpEYM\n45IkSVIjhnFJkiSpEcO4JEmS1Ihf+qMpbWj2bIb9AgFJkrSZGrcznmR1ksVJbklyYZKZ3fF/X9+L\nJlmYZL/u8TeS7Li+a0mSJEmbqkG2qaysqnlVtQ/w38CxAFX1rMkooKoOq6r7JmMtSZIkaVMy0T3j\n3wGeBJBkeffv/CSLklyc5HtJzkyyVXfueUmuTnJ911WfNXbBJLcn2TnJ3CTfT/LpJLcmuSzJjG7M\nE5N8K8lIku8k2XvDbluSJElqb+A940mmA4cC31rD6f2BpwN3dOf/NMlC4L3Ac6tqRZJ3AicC71/H\nZZ4MvLKq3pDkS8BLgS8AnwKOraofJzkA+CTwB4PWrk3XyMidJKe2LkOSpA1WdUrrEjQFDRLGZyRZ\n3D3+DvCZNYy5tqp+CpDkfOAgYBW9gH5VEoBtgKvHudZtVTV6rRFgbtdNfxZwYbcOwLYD1C1JkiRN\naYOE8ZVVNW+cMbWG5wG+XVWvnEA9D/Y9Xg3MoLeV5r4BapAkSZI2KZP1OeP7J9mz2yv+CuBK4Brg\n2UlG95jPTPKUiS5cVQ8AtyV5WbdOkvzuJNUtSZIkNTNZYfxq4DTgFuA24OKqugc4Cjg/yU30wvn6\nvvHy1cDrktwI3Aq8aIMrliRJkhpL1dgdJhNcIJkPnFRVL5iUiqQ+yZyCY1qXIUnSBvMNnFuWJCNV\ntd944yarMy5JkiRpggb+aMO1qaqFwMINrkSSJEnawtgZlyRJkhrZ4M64tDENDc1heNg9dpIkafNk\nZ1ySJElqxDAuSZIkNWIYlyRJkhoxjEuSJEmNGMYlSZKkRgzjkiRJUiOGcUmSJKkRw7gkSZLUiGFc\nU9tdI60rkCRJ2mgM45IkSVIjhnFJkiSpEcO4JEmS1IhhXJIkSWrEMC5JkiQ1YhiXJEmSGjGMS5Ik\nSY0YxjW17TrUugJJkqSNxjAuSZIkNWIYlyRJkhoxjEuSJEmNGMYlSZKkRqa3LkBal5Fly8jCha3L\nkCRJm4maP791Cb/BzrgkSZLUiGFckiRJasQwLkmSJDViGJckSZIaMYxLkiRJjRjGJUmSpEYM45Ik\nSVIjfs64prSh2bMZnmKfBypJkjRZ7IxLkiRJjYwbxpOsTrI4yS1JLkwyszv+7+t70SQLk+zXPf5G\nkh3Xdy1JkiRpUzVIZ3xlVc2rqn2A/waOBaiqZ01GAVV1WFXdNxlrSZIkSZuSiW5T+Q7wJIAky7t/\n5ydZlOTiJN9LcmaSrbpzz0tydZLru676rLELJrk9yc5J5ib5fpJPJ7k1yWVJZnRjnpjkW0lGknwn\nyd4bdtuSJElSewO/gTPJdOBQ4FtrOL0/8HTgju78nyZZCLwXeG5VrUjyTuBE4P3ruMyTgVdW1RuS\nfAl4KfAF4FPAsVX14yQHAJ8E/mDQ2rXpGhm5k+TU1mVIkrTBqk5pXYKmoEHC+Iwki7vH3wE+s4Yx\n11bVTwGSnA8cBKyiF9CvSgKwDXD1ONe6rapGrzUCzO266c8CLuzWAdh2gLolSZKkKW2QML6yquaN\nM6bW8DzAt6vqlROo58G+x6uBGfS20tw3QA2SJEnSJmWyPtpw/yR7dnvFXwFcCVwDPDvJ6B7zmUme\nMtGFq+oB4LYkL+vWSZLfnaS6JUmSpGYmK4xfDZwG3ALcBlxcVfcARwHnJ7mJXjhf3zdevhp4XZIb\ngVuBF21wxZIkSVJjqRq7w2SCCyTzgZOq6gWTUpHUJ5lTcEzrMiRJ2mC+gXPLkmSkqvYbb5zfwClJ\nkiQ1MvBHG65NVS0EFm5wJZIkSdIWxs64JEmS1MgGd8aljWloaA7Dw+6xkyRJmyc745IkSVIjhnFJ\nkiSpEcO4JEmS1IhhXJIkSWrEMC5JkiQ1YhiXJEmSGjGMS5IkSY0YxiVJkqRGDOOSJElSI4ZxSZIk\nqRHDuCRJktSIYVySJElqxDAuSZIkNWIYlyRJkhoxjEuSJEmNGMYlSZKkRgzjkiRJUiOGcUmSJKkR\nw7gkSZLUiGFckiRJamR66wKkdRlZtowsXLhR1q758zfKupIkSYOyMy5JkiQ1YhiXJEmSGjGMS5Ik\nSY0YxiVJkqRGDOOSJElSI4ZxSZIkqRHDuCRJktSIYVySJElqxC/90ZQ2NHs2w345jyRJ2kzZGZck\nSZIaGTeMJ1mdZHGSG5Ncn+RZj0Rha6llbpJbusfzk3yte/zCJO/qHi9I8qskj+mbt7zv8ZS5H0mS\nJG3ZBumMr6yqeVX1u8C7gb8ddPH0bPTue1VdUlWn9R1aCrxjLcPX+34kSZKkyTTRPePbA78cfZLk\nL4CXA9sCF1fVKUnmAt8E/hU4EHhxkluBjwIvAFYCL6qqu5I8ATgb2AW4Bzi6qn6W5HPA16rqy911\nllfVrLUVleQoYL+qekt36GzgqCQfqqr/GvR+NPWMjNxJcmrrMiRJekRVndK6BD1CBulaz+i2dfwA\nOAv4a4AkzwOeDOwPzAOGkvx+N+epwOer6v9U1R3Ao4Brum70IuAN3bgzunG/A5wHfGyS7ms5vUD+\ntkHvR5IkSXqkTWSbyt7A84HPJwnwvO7nBuB6YG964Rzgjqq6pm+N/wa+1j0eAeZ2jw8Evtg9Phc4\naD3vY00+BhyZZPsxx9d2P5IkSdIjakLbVKrq6iQ709tWEuBvq+of+8d021RWjJn6UFVV93j1Oq47\nOuZhuhcKXVDeZiJ1drXel+SLwHHrGNN/P3dP9BqSJEnShpjQmyuT7A1MA+4FLgX+PMms7tzu/Z9g\nMqB/B47oHr8auLJ7fDsw1D1+EbD1BNcddTpwDGsJ/2PuR5IkSXpEDdIZn5Fkcfc4wJFVtRq4LMnT\ngKu7XR7LgdfQ63wP6njg7O6NoPcAR3fHPw38S5JrgSv4/3faB1JVS5NcDJwwwP1IkiRJj6j87+4R\naepJ5lTvjxuSJG05/DSVTV+Skarab7xxfgOnJEmS1IhhXJIkSWpkol/6Iz2ihobmMDzsn+okSdLm\nyc64JEmS1IhhXJIkSWrEMC5JkiQ1YhiXJEmSGjGMS5IkSY0YxiVJkqRGDOOSJElSI4ZxSZIkqRHD\nuCRJktSIYVySJElqxDAuSZIkNWIYlyRJkhoxjEuSJEmNGMYlSZKkRgzjkiRJUiOGcUmSJKkRw7gk\nSZLUiGFckiRJasQwLkmSJDViGJckSZIaMYxLkiRJjUxvXYC0LiPLlpGFCwceX/Pnb7RaJEmSJpud\ncUmSJKkRw7gkSZLUiGFckiRJasQwLkmSJDViGJckSZIaMYxLkiRJjRjGJUmSpEYM45IkSVIjfumP\nprSh2bMZ9ot8JEnSZmrczniSSnJu3/PpSe5J8rUB5i7v/p2b5FV9x/dL8rH1LXoQSV6Y5F3jjDkq\nyRnd4wVJfpXkMX3nl/c9Xp1kcZIbk1yf5Fkbr3pJkiRtCQbZprIC2CfJjO75HwG/mOB15gL/E8ar\nariqjp/gGhNSVZdU1WkTnLYUeMdazq2sqnlV9bvAu4G/3aACJUmStMUbdM/4N4E/6R6/Ejh/9ETX\nUT6p7/ktSeaOmX8acHDXWT4hyfzRzno3/+wkC5P8NMnxfWud2K13S5K3d8fmJvlBkrO64+cleW6S\nq5L8OMn+3bj+rvfhSb6b5IYklyfZdS33eTbwiiSPHuf3sT3wy3HGSJIkSes06J7xfwJO7gL079AL\nrQdP4DrvAk6qqhcAJJk/5vzewCHAbOCHSf6hu87RwAFAgO8m+Td6IfhJwMuANwLX0eu6HwS8EPgr\n4MVj1r8SeGZVVZLXA3/Jmjvgy7t7extwyphzM5IsBrYDdgP+YAL3r/U0MnInyamty5AkaaOpGhs5\ntCUZKIxX1U1dt/uVwDc2Qh1fr6oHgQeT3A3sSi9cX1xVKwCSXETvBcAlwG1VdXN3/Fbgii5o30xv\nS8xYjwMuSLIbsA1w2zpq+RiwOMnfjzm+sqrmddc8EPh8kn2qqtbvliVJkrSlm8hHG14CfJi+LSqd\nh8ess9161PFg3+PV9F4kZMDxv+57/mvW/ALj48AZVfUM4Jh11VhV9wFfBI5bx5irgZ2BXdZRoyRJ\nkrROEwnjZwPvH+1I97kd2Bcgyb7AnmuYu4zeFpSJWAS8OMnMJI8CXgJ8Z4JrjNqB/33T6ZEDjD+d\nXmhf418OkuwNTAPuXc96JEmSpMHDeFUtqaqPruHUV4BHd/up3wT8aA1jbgIe7j4W8IQBr3c98Dng\nWuC7wFlVdcOg9Y6xALgwyXfofWLKeNdeClwMbNt3eEb3BtTFwAXAkVW1ej3rkSRJkohbnjWVJXOq\n90cKSZI2T76Bc/OUZKSq9htv3ES2qUiSJEmaRIZxSZIkqRHDuCRJktTIoF/6IzUxNDSH4WH30kmS\npM2TnXFJkiSpEcO4JEmS1IhhXJIkSWrEMC5JkiQ1YhiXJEmSGjGMS5IkSY0YxiVJkqRGDOOSJElS\nI4ZxSZIkqRHDuCRJktSIYVySJElqxDAuSZIkNWIYlyRJkhoxjEuSJEmNGMYlSZKkRgzjkiRJUiOG\ncUmSJKkRw7gkSZLUiGFckiRJasQwLkmSJDViGJckSZIamd66AGldRpYtIwsXti5DkrSZq/nzW5eg\nLZSdcUmSJKkRw7gkSZLUiGFckiRJasQwLkmSJDViGJckSZIaMYxLkiRJjRjGJUmSpEb8nHFNaUOz\nZzPsZ79KkqTNlJ1xSZIkqZGBwniS9yS5NclNSRYnOSDJ9CQfTPLj7tjiJO/pm7O6O3ZrkhuTnJhk\nq77z+ydZlOSHSX6Q5KwkM5McleSMybrBJN9IsmP3+Pgk309yXpIXJnnXZF1HkiRJmqhxt6kkORB4\nAbBvVT2YZGdgG+ADwGOBZ1TVqiSzgXf0TV1ZVfO6NR4DfBHYATglya7AhcARVXV1kgAvBWZP4r0B\nUFWH9T09Dji0qm7rnl8y6DpJplfVw5NanCRJkrZog3TGdwOWVtWDAFW1FLgPeAPw1qpa1R1fVlUL\n1rRAVd0NvBF4Sxe83wycU1VXd+erqr5cVXf1z0tyeJLvJrkhyeVdiCfJc/q68TckmZ1kt67TvjjJ\nLUkO7sbenmTnJGcCewGXJDmhvwOfZJckX0lyXffz7O74giSfSnIZ8PkJ/F4lSZKkcQ3yBs7LgJOT\n/Ai4HLgA+CXws6paNuiFquqn3TaVxwD7AOcMMO1K4JlVVUleD/wlve77ScCbq+qqJLOAVfTC/qVV\n9TdJpgEzx1z/2CTPBw6pqqVJjuo7/VHgI1V1ZZI9gEuBp3XnhoCDqmrloPeqyTMycifJqa3LkCRp\ng1Wd0roETUHjhvGqWp5kCDgYOIReGP9g/5gkRwNvA3YCnlVVP1/LcplgfY8DLkiyG72tMaPbS64C\nTk9yHnBRVS1Jch1wdpKtgX+uqsUTuM5zgaf3mvYAbN9tuwG4xCAuSZKkjWGgN3BW1eqqWli9l3Rv\nAQ4H9hgNrFX12W5/+P3AtDWtkWQvYDVwN3ArvY7zeD4OnFFVzwCOAbbrrnca8HpgBnBNkr2rahHw\n+8AvgHOTvHaQe+tsBRxYVfO6n937uv4rJrCOJEmSNLBxw3iSpyZ5ct+hecAPgc8AZyTZrhs3jV73\nek1r7AKcSS9YF3AGcGSSA/rGvCbJY8dM3YFeuAY4sm/sE6vq5qr6EDAM7J3kCcDdVfXprrZ9x7u3\nPpfRe5Exuv68CcyVJEmS1ssge8ZnAR/vPh7wYeAn9PZn3w/8NXBLkmXASnr7wO/s5s1IshjYupt3\nLnA6QFXdleQI4MPdJ638GlgEXDTm2guAC5P8ArgG2LM7/vYkh9DrtH8P+CZwBPAXSR4ClgMT6Ywf\nD3wiyU30fieLgGMnMF+SJEmasPQa1dLUlMyp3g4lSZI2bb6Bc8uSZKSq9htvnN/AKUmSJDViGJck\nSZIaGWTPuNTM0NAchof9s54kSdo82RmXJEmSGjGMS5IkSY0YxiVJkqRGDOOSJElSI4ZxSZIkqRHD\nuCRJktSIYVySJElqxDAuSZIkNWIYlyRJkhoxjEuSJEmNGMYlSZKkRgzjkiRJUiOGcUmSJKkRw7gk\nSZLUiGFckiRJasQwLkmSJDViGJckSZIaMYxLkiRJjRjGJUmSpEYM45IkSVIjhnFJkiSpkemtC5DW\nZWTZMrJwYesyJElbuJo/v3UJ2kzZGZckSZIaMYxLkiRJjRjGJUmSpEYM45IkSVIjhnFJkiSpEcO4\nJEmS1IhhXJIkSWrEMC5JkiQ14pf+aEobmj2bYb9oQZIkbaYG6owneU+SW5PclGRxkgOSTE/ywSQ/\n7o4tTvKevjmru2O3JrkxyYlJtuo7v3+SRUl+mOQHSc5KMjPJUUnOmKwbTPKNJDt2j49P8v0k5yV5\nYZJ3TdZ1JEmSpIkatzOe5EDgBcC+VfVgkp2BbYAPAI8FnlFVq5LMBt7RN3VlVc3r1ngM8EVgB+CU\nJLsCFwJHVNXVSQK8FJg9ifcGQFUd1vf0OODQqrqte37JoOskmV5VD09qcZIkSdqiDdIZ3w1YWlUP\nAlTVUuA+4A3AW6tqVXd8WVUtWNMCVXU38EbgLV3wfjNwTlVd3Z2vqvpyVd3VPy/J4Um+m+SGJJd3\nIZ4kz+nrxt+QZHaS3bpO++IktyQ5uBt7e5Kdk5wJ7AVckuSE/g58kl2SfCXJdd3Ps7vjC5J8Ksll\nwOcn8HuVJEmSxjXInvHLgJOT/Ai4HLgA+CXws6paNuiFquqn3TaVxwD7AOcMMO1K4JlVVUleD/wl\nve77ScCbq+qqJLOAVfTC/qVV9TdJpgEzx1z/2CTPBw6pqqVJjuo7/VHgI1V1ZZI9gEuBp3XnhoCD\nqmrloPeqyTMycifJqa3LkCRtgapOaV2CtgDjhvGqWp5kCDgYOIReGP9g/5gkRwNvA3YCnlVVP1/L\ncplgfY8DLkiyG72tMaPbS64CTk9yHnBRVS1Jch1wdpKtgX+uqsUTuM5zgaf3mvYAbN9tuwG4xCAu\nSZKkjWGgN3BW1eqqWli9l4hvAQ4H9hgNrFX12W5/+P3AtDWtkWQvYDVwN3ArvY7zeD4OnFFVzwCO\nAbbrrnca8HpgBnBNkr2rahHw+8AvgHOTvHaQe+tsBRxYVfO6n937uv4rJrCOJEmSNLBxw3iSpyZ5\nct+hecAPgc8AZyTZrhs3jV73ek1r7AKcSS9YF3AGcGSSA/rGvCbJY8dM3YFeuAY4sm/sE6vq5qr6\nEDAM7J3kCcDdVfXprrZ9x7u3PpfRe5Exuv68CcyVJEmS1ssge8ZnAR/vPh7wYeAn9PZn3w/8NXBL\nkmXASnr7wO/s5s1IshjYupt3LnA6QFXdleQI4MPdJ638GlgEXDTm2guAC5P8ArgG2LM7/vYkh9Dr\ntH8P+CZwBPAXSR4ClgMT6YwfD3wiyU30fieLgGMnMF+SJEmasPQa1dLUlMyp3g4lSZIeWb6BUxsi\nyUhV7TfeuIH2jEuSJEmafIZxSZIkqRHDuCRJktTIIG/glJoZGprD8LB79iRJ0ubJzrgkSZLUiGFc\nkiRJasQwLkmSJDViGJckSZIaMYxLkiRJjRjGJUmSpEYM45IkSVIjhnFJkiSpEcO4JEmS1IhhXJIk\nSWrEMC5JkiQ1YhiXJEmSGjGMS5IkSY0YxiVJkqRGDOOSJElSI4ZxSZIkqRHDuCRJktSIYVySJElq\nxDAuSZIkNWIYlyRJkhoxjEuSJEmNTG9dgLQuI8uWkYULW5chSdLAav781iVoE2JnXJIkSWrEMC5J\nkiQ1YhiXJEmSGjGMS5IkSY0YxiVJkqRGDOOSJElSI4ZxSZIkqRHDuCRJktSIX/qjKW1o9myG/fIE\nSZK0mRq3M55k+RqOHZvktRunpN+4zp8nuTnJTUluSfKiJEclOX/MuJ2T3JNk2yRbJzktyY+7Odcm\nOXRj1ypJkiRN1Hp1xqvqzMkupF+SAI8H3gPsW1X3J5kF7ALcC3w4ycyq+lU35c+AS6rqwSSnAbsB\n+3TPdwWeszHrlSRJktbHeu0ZT7IgyUnd44VJPtR1oH+U5ODu+LQkf5fkuq6zfUx3fFaSK5Jc33W9\nX9Qdn5vk+0k+CVwP7AksA5YDVNXyqrqtqh4AFgGH95V0BHB+kpnAG4C3VtWD3by7qupL63OfkiRJ\n0sY0WXvGp1fV/kkOA04Bngu8Dri/qn4vybbAVUkuA34OvKSqHkiyM3BNkku6dZ4KHF1VxyWZBtwF\n3JbkCuCiqvpqN+584FXABUnmAE8B/hX4beBnXWDXZmBk5E6SU1uXIUnSwKpOaV2CNiGT9WkqF3X/\njgBzu8fPA16bZDHwXWAn4MlAgA8muQm4HNgd2LWbc0dVXQNQVauB59PbgvIj4CNJFnTjvgYclGR7\n4OXAl7vxkiRJ0iZjsjrjD3b/ru5bM/S2i1zaPzDJUfT2fg9V1UNJbge2606v6B9bVQVcC1yb5NvA\nZ4EFVbUyybeAl9DbonJCN+XvymbTAAAgAElEQVQnwB5JZlfVskm6N0mSJGmj2JifM34p8KYkWwMk\neUqSRwE7AHd3QfwQ4AlrmpxkTpJ9+w7NA+7oe34+cCK9rvpoN/1XwGeAjyXZpltntySvmdxbkyRJ\nkjbcIJ3xmUmW9D0/fcC1z6K3ZeX67tNR7gFeDJwHfDXJMLAY+MFa5m9N71NT5gCruvnH9p2/DDgH\n+EzXQR/1XuADwPeSrKLXbT95wJolSZKkR0x+M8dKU0syp+CY1mVIkjQw38ApgCQjVbXfeOM25jYV\nSZIkSetgGJckSZIaMYxLkiRJjUzWRxtKG8XQ0ByGh917J0mSNk92xiVJkqRGDOOSJElSI4ZxSZIk\nqRHDuCRJktSIYVySJElqxDAuSZIkNWIYlyRJkhoxjEuSJEmNGMYlSZKkRgzjkiRJUiOGcUmSJKkR\nw7gkSZLUiGFckiRJasQwLkmSJDViGJckSZIaMYxLkiRJjRjGJUmSpEYM45IkSVIjhnFJkiSpEcO4\nJEmS1IhhXJIkSWpkeusCpHUZWbaMLFzYugxNQTV/fusSJEnaYHbGJUmSpEYM45IkSVIjhnFJkiSp\nEcO4JEmS1IhhXJIkSWrEMC5JkiQ1YhiXJEmSGvFzxjWlDc2ezbCfJy1JkjZTdsYlSZKkRgYK40ne\nk+TWJDclWZzkgCTTk3wwyY+7Y4uTvKdvzuru2K1JbkxyYpKt+s7vn2RRkh8m+UGSs5LMTHJUkjMm\n6waTfCPJjt3j45N8P8l5SV6Y5F2TdR1JkiRposbdppLkQOAFwL5V9WCSnYFtgA8AjwWeUVWrkswG\n3tE3dWVVzevWeAzwRWAH4JQkuwIXAkdU1dVJArwUmD2J9wZAVR3W9/Q44NCquq17fsmg6ySZXlUP\nT2pxkiRJ2qIN0hnfDVhaVQ8CVNVS4D7gDcBbq2pVd3xZVS1Y0wJVdTfwRuAtXfB+M3BOVV3dna+q\n+nJV3dU/L8nhSb6b5IYkl3chniTP6evG35BkdpLduk774iS3JDm4G3t7kp2TnAnsBVyS5IT+DnyS\nXZJ8Jcl13c+zu+MLknwqyWXA5yfwe5UkSZLGNcgbOC8DTk7yI+By4ALgl8DPqmrZoBeqqp9221Qe\nA+wDnDPAtCuBZ1ZVJXk98Jf0uu8nAW+uqquSzAJW0Qv7l1bV3ySZBswcc/1jkzwfOKSqliY5qu/0\nR4GPVNWVSfYALgWe1p0bAg6qqpWD3qsmz8jInSSnti5DkqRJVXVK6xI0RYwbxqtqeZIh4GDgEHph\n/IP9Y5IcDbwN2Al4VlX9fC3LZYL1PQ64IMlu9LbGjG4vuQo4Pcl5wEVVtSTJdcDZSbYG/rmqFk/g\nOs8Fnt5r2gOwfbftBuASg7gkSZI2hoHewFlVq6tqYfVexr0FOBzYYzSwVtVnu/3h9wPT1rRGkr2A\n1cDdwK30Os7j+ThwRlU9AzgG2K673mnA64EZwDVJ9q6qRcDvA78Azk3y2kHurbMVcGBVzet+du/r\n+q+YwDqSJEnSwMYN40memuTJfYfmAT8EPgOckWS7btw0et3rNa2xC3AmvWBdwBnAkUkO6BvzmiSP\nHTN1B3rhGuDIvrFPrKqbq+pDwDCwd5InAHdX1ae72vYd7976XEbvRcbo+vMmMFeSJElaL4PsGZ8F\nfLz7eMCHgZ/Q2599P/DXwC1JlgEr6e0Dv7ObNyPJYmDrbt65wOkAVXVXkiOAD3eftPJrYBFw0Zhr\nLwAuTPIL4Bpgz+7425McQq/T/j3gm8ARwF8keQhYDkykM3488IkkN9H7nSwCjp3AfEmSJGnC0mtU\nS1NTMqd6O5QkSdp8+AbOzV+Skarab7xxfgOnJEmS1IhhXJIkSWpkkD3jUjNDQ3MYHvZPeZIkafNk\nZ1ySJElqxDAuSZIkNWIYlyRJkhoxjEuSJEmNGMYlSZKkRgzjkiRJUiOGcUmSJKkRw7gkSZLUiGFc\nkiRJasQwLkmSJDViGJckSZIaMYxLkiRJjRjGJUmSpEYM45IkSVIjhnFJkiSpEcO4JEmS1IhhXJIk\nSWrEMC5JkiQ1YhiXJEmSGjGMS5IkSY0YxiVJkqRGprcuQFqXkWXLyMKFrcuQ/kfNn9+6BEnSZsTO\nuCRJktSIYVySJElqxDAuSZIkNWIYlyRJkhoxjEuSJEmNGMYlSZKkRgzjkiRJUiOGcUmSJKkRv/RH\nU9rQ7NkM+yUrkiRpMzVuZzzJ8jUcOzbJazdOSb9xnT9PcnOSm5LckuRFSY5Kcv6YcTsnuSfJtkm2\nTnJakh93c65NcujGrlWSJEmaqPXqjFfVmZNdSL8kAR4PvAfYt6ruTzIL2AW4F/hwkplV9atuyp8B\nl1TVg0lOA3YD9ume7wo8Z2PWK0mSJK2P9doznmRBkpO6xwuTfKjrQP8oycHd8WlJ/i7JdV1n+5ju\n+KwkVyS5vut6v6g7PjfJ95N8Erge2BNYBiwHqKrlVXVbVT0ALAIO7yvpCOD8JDOBNwBvraoHu3l3\nVdWX1uc+JUmSpI1psvaMT6+q/ZMcBpwCPBd4HXB/Vf1ekm2Bq5JcBvwceElVPZBkZ+CaJJd06zwV\nOLqqjksyDbgLuC3JFcBFVfXVbtz5wKuAC5LMAZ4C/Cvw28DPusCuzcDIyJ0kp7YuQ5KkjaLqlNYl\nqLHJ+jSVi7p/R4C53ePnAa9Nshj4LrAT8GQgwAeT3ARcDuwO7NrNuaOqrgGoqtXA8+ltQfkR8JEk\nC7pxXwMOSrI98HLgy914SZIkaZMxWZ3xB7t/V/etGXrbRS7tH5jkKHp7v4eq6qEktwPbdadX9I+t\nqgKuBa5N8m3gs8CCqlqZ5FvAS+htUTmhm/ITYI8ks6tq2STdmyRJkrRRbMzPGb8UeFOSrQGSPCXJ\no4AdgLu7IH4I8IQ1TU4yJ8m+fYfmAXf0PT8fOJFeV320m/4r4DPAx5Js062zW5LXTO6tSZIkSRtu\nkM74zCRL+p6fPuDaZ9HbsnJ99+ko9wAvBs4DvppkGFgM/GAt87em96kpc4BV3fxj+85fBpwDfKbr\noI96L/AB4HtJVtHrtp88YM2SJEnSIya/mWOlqSWZU3BM6zIkSdoofAPn5ivJSFXtN964jblNRZIk\nSdI6GMYlSZKkRgzjkiRJUiOT9dGG0kYxNDSH4WH300mSpM2TnXFJkiSpEcO4JEmS1IhhXJIkSWrE\nMC5JkiQ1YhiXJEmSGjGMS5IkSY0YxiVJkqRGDOOSJElSI37pj6a2u0bg79O6CkmStLl4R7Wu4DfY\nGZckSZIaMYxLkiRJjRjGJUmSpEYM45IkSVIjhnFJkiSpEcO4JEmS1IhhXJIkSWrEzxnX1LbrELxj\nuHUVkiRJG4WdcUmSJKkRw7gkSZLUiGFckiRJasQwLkmSJDXiGzg1pY0sW0YWLvyNYzV/fpNaJEmS\nJpudcUmSJKkRw7gkSZLUiGFckiRJasQwLkmSJDViGJckSZIaMYxLkiRJjRjGJUmSpEYM45IkSVIj\nfumPprSh2bMZ9kt+JEnSZmrczniS5X2PD0vy4yR7JFmQ5FdJHrOmsetY7xtJdhxnzMIk+63h+FFJ\nzhjvGusjyUlJfpDkliQ3JnntumpZz2vsl+Rj3eNtk1yeZHGSVyQ5K8nTJ+M6kiRJ2jQM3BlP8ofA\nx4HnVdXPkgAsBd4BvHPQdarqsIkWORnSKzhV9es1nDsW+CNg/6p6IMkOwIsnu4aqGgaGu6f/B9i6\nquZ1zy+YyFpJplXV6smsT5IkSY+sgfaMJzkY+DTwJ1X1H32nzgZekeTRa5jzmiTXdp3ff0wyrTt+\ne5Kdu8fv67rR305yfpKT+pZ4WTf/R931Rz0+ybeS/DDJKX3XO7Hrat+S5O3dsblJvp/kk8D13dzP\ndWNuTnJCN/2vgOOq6gGAqrq/qs5Zwz39Q5LhJLcmObXv+GlJvpfkpiQf7o69rK/Lvqg7Nj/J17q/\nJnwBmNf9fp7Y34FP8rwkVye5PsmFSWb1/e5OTnIl8LJx/8NJkiRpShukM74t8C/A/Kr6wZhzy+kF\n8rcB/cH4acArgGdX1UNdGH418Pm+MfsBL6XXIZ5OLyyP9NdWVfsnOaxb+7nd8f2BfYBfAdcl+TpQ\nwNHAAUCA7yb5N+CXwFOBo6vquCRDwO5VtU9Xw45JZgOzx7zIWJv3VNV/dS8srkjyO8AS4CXA3lVV\nfVtwTgb+uKp+MXZbTlXdneT1wElV9YKultHfy87Ae4HnVtWKJO8ETgTe301fVVUHDVDrZmFk5E76\nXvdIkjYzVaeMP0jajA3SGX8I+HfgdWs5/zHgyCTb9x37Q2CIXlhe3D3fa8y8g4B/qaqVVbUM+OqY\n8xd1/44Ac/uOf7uq7q2qld2Yg7qfi6tqRVUt746PdtPvqKprusc/BfZK8vEkzwceoBfea52/gf/1\n8iTXAzcAvw08vVtjFXBWkj+l9yIB4Crgc0neAEwbcH2AZ3brXtX97o4EntB3fkLbWSRJkjR1DRLG\nfw28HPi9JH819mRV3Qd8ETiu73CAc6pqXvfz1KpaMGZqxrnug92/q/nNDv7Y4FzjrLWir9ZfAr8L\nLATeDJzVbU1ZkWTsi4XfLDbZEzgJ+MOq+h3g68B2VfUwvW79V+jtM/9Wd61j6XW4Hw8sTrLTutbv\nvxS9Fxyjv7unV1X/C6EVa5soSZKkTctAe8ar6lfAC4BXJ1lTh/x04Bj+NzRfAfzZ6CetJHl0kieM\nmXMlcHiS7bo90X8yYM1/1K03g174vQpYBLw4ycwkj6K3beQ7Yyd2W0C2qqqvAO8D9u1O/S3widHu\nfpLtk7xxzPTt6QXh+5PsChzajZ0F7FBV3wDeDszrjj+xqr5bVSfTe6Pr4we8v2uAZyd5UrfOzCRP\nGXCuJEmSNiEDf5pKt1f6+cCiJEvHnFua5GLghO7595K8F7gsyVb0trq8Gbijb851SS4BbuyODwP3\nD1DKlcC5wJOAL3afUEKSzwHXdmPOqqobkswdM3d34LNdTQDv7v79B2AWvW01D3X1/v2Ye7wxyQ3A\nrfS2u1zVnZoN/EuS7eh1tUffFPp3SZ7cHbuiu8/njHdzVXVPkqOA85Ns2x1+L/Cj8eZKkiRp05Kq\nQbdLb4SLJ7OqanmSmfS622+squubFaQpJ5lTvT+6SJI2R76BU5urJCNVNe531bT+Bs5PpfdFN9vR\n22NuEJckSdIWo2kYr6pXtby+JEmS1NJAb+CUJEmSNPlab1OR1mloaA7Dw+4nlCRJmyc745IkSVIj\nhnFJkiSpEcO4JEmS1IhhXJIkSWrEMC5JkiQ1YhiXJEmSGjGMS5IkSY0YxiVJkqRGDOOSJElSI4Zx\nSZIkqRHDuCRJktSIYVySJElqxDAuSZIkNTK9dQGSJEmamIceeoglS5awatWq1qVs8bbbbjse97jH\nsfXWW6/XfMO4JEnSJmbJkiXMnj2buXPnkqR1OVusquLee+9lyZIl7Lnnnuu1httUJEmSNjGrVq1i\np512Mog3loSddtppg/5CYRiXJEnaBBnEp4YN/e9gGJckSZIacc+4JEnSJi45dVLXqzplUtfT2tkZ\nlyRJUjMPP/xw6xKasjOuKW1k2TKycGHrMiRJY9T8+a1LUEMrVqzg5S9/OUuWLGH16tW8733vY6+9\n9uJtb3sbK1asYNttt+WKK65g66235k1vehPDw8NMnz6d008/nUMOOYTPfe5zfP3rX2fVqlWsWLGC\n/6+9ew+vqrzyOP5dhHAJoCBOZ0KZAlLBNBcCIT4gAmIlBHUcaWfUIipaAQEv1ZIC1hF1WrWlF4kX\nUCtSlVgs2GmnMDTDKOOFMEBoFChISs1QwZaLBhMbGYJr/jibPDE5JDmQnJ00v8/znId9znn3u9fJ\nIicrb9be55VXXmHhwoW89NJLHD16lEmTJnH//c272t9aqRgXERERkZisXbuWPn36sHr1agCOHDnC\n0KFDWbFiBdnZ2Xz00Ud07dqVRYsWAbBt2zZ27dpFTk4Ou3fvBqCoqIi3336bs846i8LCQkpLS9m0\naRPuzhVXXMFrr73GmDFjQnuN8aI2FRERERGJSXp6OuvWrWPu3Lm8/vrr7N27l+TkZLKzswE444wz\n6NixI2+88QbXXXcdAOeddx79+vWrKcbHjx/PWWedBUBhYSGFhYUMHTqUYcOGsWvXLkpLS8N5cXGm\nlXERERERicmgQYMoLi5mzZo1zJ8/n5ycnKiX+HP3k87RrVu3z4ybP38+M2bMaJF4WzOtjIuIiIhI\nTPbv309SUhJTpkxhzpw5bNy4kf3797N582YAKioqqK6uZsyYMSxfvhyA3bt3s3fvXgYPHlxvvgkT\nJrB06VIqKysB2LdvHwcOHIjfCwqRVsZFRERE2rh4X4pw27Zt5OXl0aFDBxITE1m8eDHuzm233UZV\nVRVdu3Zl3bp1zJo1i1tuuYX09HQ6duzIsmXL6Ny5c735cnJy2LlzJyNHjgSge/fuvPDCC3zuc5+L\n6+sKgzX05wORsNngwc6TT4YdhoiI1KGrqYRr586dpKSkhB2GBKLlw8yK3X14Y/uqTUVEREREJCQq\nxkVEREREQqKecWnVsnr0YIv+FCoiIiJ/pbQyLiIiIiISkkaLcTOrPN2DmFkfM1vZwPM9zWxWU8cH\nY9ab2Ttm9paZbTazzNONszmZ2QNmdknYcYiIiIhI6xWXlXF33+/u/9TAkJ7ArBjGn3Ctuw8BngAW\nnmaYAJhZs7TuuPu97r6uOeYSERERkb9Op1R4mlk/YCnwN8BB4EZ332tmA4HlQALwH8Bd7t7dzPoD\nv3b3NDNLBZ4FOhH5ZeCrwL8CA82sBPhP4PFa4xOA7wETAAeedvdH64RUBOTVii8HuB/oDOwJ4qs0\ns0uBHwGHgK3AOe5+uZndB/QB+gOHzOw64GHgomCOx939STNLBlYAZwRfu5nABuAZYHgQ31J3/7GZ\nLQtew0oz+zLwg2CfzcBMdz9qZmXAT4F/ABKBf3b3XbHmQ0RERNo3W7++Wedr7NKV5eXlFBQUMGvW\nrAbHRXPppZdSUFBAz549Tzrm3nvvZcyYMVxyyek3GTz44IPcfffdNfcvuOACNmzYcNrzNpdTXQV+\nDHjO3X9qZjcB+cCVwCJgkbu/aGa3nGTfW4Ixy82sE5HCfR6Q5u6ZAEHxfsJ0YAAw1N2rzeysKHPm\nAv8W7Hs2cA9wibt/bGZzgbvM7PvAk8AYd3/XzF6sM0cWcKG7V5nZdOCIu2ebWWfgTTMrBL4C/Mbd\nvxv8kpAEZAKfd/e04Pif+Z9lZl2AZcCX3X23mT1HpIh/JBhyyN2HBW06c4CbT/J1a5eKi/djdn/Y\nYYiISCsT7w+5kc8qLy/niSeeiFqMHz9+nISEhJPuu2bNmkbnf+CBB04rvtrqFuOtqRCHU29TGQkU\nBNvPAxfWevznwXZB3Z0CRcDdQZHcz92rGjnWJcASd68GcPcPaj233MzeA+YCJ1bLRwBfIlJAlwA3\nAP2A84A/uPu7wbi6xfivasWSA1wf7P8/QG/gXCKr2jcGK+np7l4B/AE4x8weNbNc4KM68w4G3nX3\n3cH9nwJjaj3/cvBvMZGVeREREZFWbd68eezZs4fMzEzy8vJYv34948aNY/LkyaSnpwNw5ZVXkpWV\nRWpqKk899VTNvv379+fQoUOUlZWRkpLCtGnTSE1NJScnh6qqSCk2depUVq5cWTN+wYIFDBs2jPT0\ndHbtijQRHDx4kPHjxzNs2DBmzJhBv379OHToUL04q6qqyMzM5NprrwUin+4JsH79esaOHctVV13F\noEGDmDdvHsuXL+f8888nPT2dPXv21Bznq1/9KtnZ2WRnZ/Pmm28269eyuXrGm/wxnu5eAFwBVAG/\nMbOLG9nFGpj/WiKr5gVEWltOjP9Pd88Mbl9y968Hjzfk4zrHvK3WHAPcvdDdXyNSSO8Dnjez6939\nQ2AIsB6YDfwkSvwNORr8exxdalJERETagIcffpiBAwdSUlLCwoWR0/Y2bdrEd7/7XX73u98BsHTp\nUoqLi9myZQv5+fkcPny43jylpaXMnj2bHTt20LNnT1atWhX1eGeffTZbt25l5syZ/OAHPwDg/vvv\n5+KLL2br1q1MmjSJvXv3Ro2za9eulJSUsHz58nrPv/XWWyxatIht27bx/PPPs3v3bjZt2sTNN9/M\no49G1nnvuOMO7rzzTjZv3syqVau4+ebmbWI41WJ8A3BNsH0t8EawvZFIDzi1nv8MMzuHyAp1PvAr\nIAOoAHqc5FiFwC0nTqys26bi7seItKWMMLOUIIZRZvbFYHySmQ0CdhFZwe4f7Hp1A6/vN8BMM0sM\n5hhkZt2CXvkD7v40kT7xYUFbTAd3XwX8CzCszly7gP4n4gGuA/67gWOLiIiItDnnn38+AwYMqLmf\nn5/PkCFDGDFiBH/84x8pLS2tt8+AAQPIzIxcEC8rK4uysrKoc3/lK1+pN+aNN97gmmsi5WZubi69\nevWKOebs7GySk5Pp3LkzAwcOJCcnB4D09PSa46xbt45bb72VzMxMrrjiCj766CMqKipiPtbJNGUl\nNiloBTnhR8DtwFIzyyM4gTN47hvAC2b2TWA1cCTKfFcDU8zsGPAn4AF3/8DM3jSz7URO/Hy81vif\nAIOAt4N9nibSs14j6PP+ITDH3b9uZlOBF4N+b4B7gn7tWcBaMzsEbGrgNf+ESMvIVjOz4DVeSeSE\nzrwgjkrgeuDzwLNmduIXm/l1YvvEzG4Efh78QrEZWNLAsUVERETanG7dutVsr1+/nnXr1lFUVERS\nUhIXXXQRn3zySb19OnfuXLOdkJBQ06ZysnEJCQlUV1cD4N7kxoyTqn38Dh061Nzv0KFDzXE+/fRT\nioqK6Nq162kfL5pGi3F3P9nqebT2kn3ACHd3M7sG2BLMUQakBdsPAQ9FOc7kOg+dGF8N3BXcao+/\nqM79H9bafgXIjhLfq+5+XlBgP14rvvvqzPUpcHdwq+2nwa2uuqvhuPvUWtv/BQyNMqZ/re0tRIp9\nERERkVatR48eDa4OHzlyhF69epGUlMSuXbvYuHFjs8dw4YUX8tJLLzF37lwKCwv58MMPo45LTEzk\n2LFjJCYmntJxcnJyeOyxx8jLi1y4r6SkpGY1vzk0d49yFvBYUOyWAzc18/yna5qZ3UDksoq/JXJ1\nFREREZE2rbFLETa33r17M2rUKNLS0pg4cSKXXXbZZ57Pzc1lyZIlZGRkMHjwYEaMGNHsMSxYsICv\nfe1rrFixgrFjx5KcnEyPHvW7nqdPn05GRgbDhg2L2jfemPz8fGbPnk1GRgbV1dWMGTOGJUuar8nB\nmmOJX6SlmPVxmBF2GCIi0sq090sb7ty5k5SUlLDDCNXRo0dJSEigY8eOFBUVMXPmTEpKSkKJJVo+\nzKzY3Yc3tq+u3iGtWlZWH7Zsad9vuCIiIlLf3r17ueqqq/j000/p1KkTTz/9dNghnRIV4yIiIiLS\n5px77rn89re/DTuM09Zc1xkXEREREZEYqRgXEREREQmJinERERERkZCoGBcRERERCYlO4BQRERFp\n635ozTvfNxu+9HV5eTkFBQXMmjXrlKZ/5JFHmD59OklJSQBceumlFBQU0LNnz1Oa74SysjI2bNjA\n5MmRz5LcsmULzz33HPn5+ac1b0vSyriIiIiIxKS8vJwnnnjilPd/5JFH+Mtf/lJzf82aNaddiEOk\nGC8oKKi5P3z48FZdiIOKcRERERGJ0bx589izZw+ZmZk1HxO/cOFCsrOzycjIYMGCyGeEfPzxx1x2\n2WUMGTKEtLQ0VqxYQX5+Pvv372fcuHGMGzcOgP79+3Po0CHKyspISUlh2rRppKamkpOTQ1VVFQCb\nN28mIyODkSNHkpeXR1paWtS4Xn/9dTIzM/nxj3/M+vXrufzyywG47777uOGGG8jJyaF///68/PLL\nfOtb3yI9PZ3c3FyOHTsGQHFxMWPHjiUrK4sJEybw/vvvt+jXUsW4iIiIiMTk4YcfZuDAgZSUlLBw\n4UIKCwspLS1l06ZNlJSUUFxczGuvvcbatWvp06cPb731Ftu3byc3N5fbb7+dPn368Oqrr/Lqq6/W\nm7u0tJTZs2ezY8cOevbsyapVqwC48cYbWbJkCUVFRSQkJJw0rtGjR1NSUsKdd95Z7/k9e/awevVq\nfvnLXzJlyhTGjRvHtm3b6Nq1K6tXr+bYsWPcdtttrFy5kuLiYm666Sa+/e1vN+8Xrw71jIuIiIjI\naSksLKSwsJChQ4cCUFlZSWlpKaNHj2bOnDnMnTuXyy+/nNGjRzc614ABA8jMzAQgKyuLsrIyysvL\nqaio4IILLgBg8uTJ/PrXv445zokTJ5KYmEh6ejrHjx8nNzcXgPT0dMrKynjnnXfYvn0748ePB+D4\n8eMkJyfHfJxYqBgXERERkdPi7syfP58ZM2bUe664uJg1a9Ywf/58cnJyuPfeexucq3PnzjXbCQkJ\nVFVV4d7wCaVNdWLuDh06kJiYiJnV3K+ursbdSU1NpaioqFmO1xRqUxERERGRmPTo0YOKioqa+xMm\nTGDp0qVUVlYCsG/fPg4cOMD+/ftJSkpiypQpzJkzh61bt0bdvzG9evWiR48ebNy4EYCf/exnTYor\nVoMHD+bgwYM1xfixY8fYsWPHKc/XFFoZFxEREWnrGrkUYXPr3bs3o0aNIi0tjYkTJ7Jw4UJ27tzJ\nyJEjAejevTsvvPACv//978nLy6tZiV68eDEA06dPZ+LEiSQnJ0ftG4/mmWeeYdq0aXTr1o2LLrqI\nM888s96YjIwMOnbsyJAhQ5g6dWpN20xTderUiZUrV3L77bdz5MgRqqur+cY3vkFqampM88TCmmvZ\nX6QlDB8+3Lds2RJ2GCIiIq3Kzp07SUlJCTuMuKqsrKR79+5A5ETN999/n0WLFoUcVUS0fJhZsbsP\nb2xfrYyLiIiISKu3evVqHnroIaqrq+nXrx/Lli0LO6RmoWJcRERERFq9q6++mquvvjrsMJqdTuAU\nERERaYPUatw6nG4eVNvSr8MAAAWeSURBVIyLiIiItDFdunTh8OHDKshD5u4cPnyYLl26nPIcalMR\nERERaWP69u3Le++9x8GDB8MOpd3r0qULffv2PeX9VYyLiIiItDGJiYkMGDAg7DCkGahNRUREREQk\nJCrGRURERERComJcRERERCQk+gROadXMrAJ4J+w4pEnOBg6FHYQ0iXLVdihXbYdy1XbEK1f93P1v\nGhukEziltXunKR8lK+Ezsy3KVdugXLUdylXboVy1Ha0tV2pTEREREREJiYpxEREREZGQqBiX1u6p\nsAOQJlOu2g7lqu1QrtoO5artaFW50gmcIiIiIiIh0cq4iIiIiEhIVIyLiIiIiIRExbiIiIiISEhU\njEuozOwsM/uFmX1sZv9rZpNPMs7M7Htmdji4fd/MLN7xtmcx5CrPzLabWYWZvWtmefGOtb1raq5q\nje9kZrvM7L14xSgRseTKzIaZ2WtmVmlmfzazO+IZa3sXw3tgZzNbEuToAzP7dzP7fLzjba/M7FYz\n22JmR81sWSNj7zSzP5nZETNbamad4xTmZ6gYl7A9Dvwf8LfAtcBiM0uNMm46cCUwBMgALgdmxCtI\nAZqeKwOuB3oBucCtZnZN3KIUaHquTsgDDsQjMKmnSbkys7OBtcCTQG/gi0BhHOOUpn9f3QGMJPKz\nqg9QDjwaryCF/cB3gKUNDTKzCcA84MtAf+Ac4P6WDi5qLLqaioTFzLoBHwJp7r47eOx5YJ+7z6sz\ndgOwzN2fCu5/HZjm7iPiHHa7FEuuouybT+S95raWj1RizZWZDQDWAHcBT7t733jG257F+B74IPD3\n7n5d/COVGHO1GKhw928F9y8DfuTug+McdrtmZt8B+rr71JM8XwCUufvdwf0vA8vd/e/iF2WEVsYl\nTIOA4yfe2AJvAdFWGlKD5xobJy0jllzVCFqJRgM7WjA2+axYc/UocDdQ1dKBST2x5GoE8IGZbTCz\nA0HrwxfiEqVAbLl6BhhlZn3MLInIKvp/xCFGiU20uuJvzax3vANRMS5h6g4cqfPYEaBHE8YeAbqr\nbzxuYslVbfcReZ95tgVikuianCszmwR0dPdfxCMwqSeW76u+wA1EWiC+ALwLvNii0UltseRqN7AX\n2Ad8BKQAD7RodHIqotUV0PjPtWanYlzCVAmcUeexM4CKJow9A6h09VnFSyy5AiIn0RDpHb/M3Y+2\nYGzyWU3KVfBn9+8Dah8KTyzfV1XAL9x9s7t/QqS39QIzO7OFY5SIWHK1GOhCpLe/G/AyWhlvjaLV\nFdDAz7WWomJcwrQb6Ghm59Z6bAjRWxp2BM81Nk5aRiy5wsxuIjgxxt11hY74amquziVy0tLrZvYn\nIgVDcnBlgf5xiFNi+756G6i9+HBiW38djI9YcjWEyDlOHwQLEY8C5wcn4UrrEa2u+LO7H453ICrG\nJTTu/jGRAuABM+tmZqOAfwSejzL8OeAuM/u8mfUBvgksi1uw7VwsuTKza4EHgfHu/of4Riox5Go7\n8PdAZnC7GfhzsP3H+EXcfsX4HvgsMMnMMs0sEfgX4A13L49fxO1XjLnaDFxvZmcGuZoF7Hf3Q/GL\nuP0ys45m1gVIABLMrIuZdYwy9Dng62b2JTPrBdxDSHWFinEJ2yygK5HLqr0IzHT3HWY22swqa417\nEvh3YBuRImJ18JjET1Nz9R0if57dHFwPudLMloQQb3vWaK7cvdrd/3TiBnwAfBrcPx5e6O1Ok76v\n3P0VIifarg7GfhFo8Prx0uya+h44B/gEKAUOApcCk+IdbDt2D5G2rnnAlGD7HjP7QvDz6AsA7r6W\nSKveq8D/BrcFYQSsSxuKiIiIiIREK+MiIiIiIiFRMS4iIiIiEhIV4yIiIiIiIVExLiIiIiISEhXj\nIiIiIiIhUTEuIiIiIhISFeMiIiIiIiFRMS4iIiIiEpL/B1Ej1agvbIwRAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2a3a8d94da0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(12, 8))\n",
    "plt.title(\"Benchmark Scores\")\n",
    "plt.barh(indices, score, .2, label=\"score\", color=\"navy\")\n",
    "plt.barh(indices + .3, training_time, .2, label=\"training time\", color='c')\n",
    "plt.barh(indices + .6, testing_time, .2, label=\"testing time\", color='darkorange')\n",
    "plt.yticks(())\n",
    "plt.legend(loc='best')\n",
    "plt.subplots_adjust(left=.25)\n",
    "plt.subplots_adjust(top=.95)\n",
    "plt.subplots_adjust(bottom=.05)\n",
    "\n",
    "for i, c in zip(indices, clf_names):\n",
    "    plt.text(-.3, i, c)\n",
    "    \n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## VotingClassifier\n",
    "http://scikit-learn.org/stable/modules/ensemble.html#using-the-votingclassifier-with-gridsearch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 317,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(train['text'], train['author'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 318,
   "metadata": {},
   "outputs": [],
   "source": [
    "pipeline_tuned = Pipeline([\n",
    "    ('vect', CountVectorizer(stop_words='english',ngram_range=(1,2), max_features=None, max_df=0.75)),\n",
    "    ('tfidf', TfidfTransformer(norm='l2', use_idf=True)),\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 319,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train_transformed = pipeline_tuned.fit_transform(X_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 320,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.ensemble import VotingClassifier\n",
    "\n",
    "clf1 = SGDClassifier(alpha=1e-05, n_iter=50, penalty='l2', loss='log')\n",
    "clf2 = MultinomialNB(alpha=0.01)\n",
    "eclf = VotingClassifier(estimators=[('svc', clf1), ('mnb', clf2)], voting='soft')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 321,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\anoop\\Anaconda3\\lib\\site-packages\\sklearn\\linear_model\\stochastic_gradient.py:117: DeprecationWarning: n_iter parameter is deprecated in 0.19 and will be removed in 0.21. Use max_iter and tol instead.\n",
      "  DeprecationWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "VotingClassifier(estimators=[('svc', SGDClassifier(alpha=1e-05, average=False, class_weight=None, epsilon=0.1,\n",
       "       eta0=0.0, fit_intercept=True, l1_ratio=0.15,\n",
       "       learning_rate='optimal', loss='log', max_iter=None, n_iter=50,\n",
       "       n_jobs=1, penalty='l2', power_t=0.5, random_state=None,\n",
       "       shuffle=True, tol=None, verbose=0, warm_start=False)), ('mnb', MultinomialNB(alpha=0.01, class_prior=None, fit_prior=True))],\n",
       "         flatten_transform=None, n_jobs=1, voting='soft', weights=None)"
      ]
     },
     "execution_count": 321,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "eclf.fit(X_train_transformed, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 322,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_test_transformed = pipeline_tuned.transform(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 323,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = eclf.predict(X_test_transformed)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 324,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred_prob = eclf.predict_proba(X_test_transformed)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 325,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy score: 0.835137895812\n",
      "Log loss: 0.435931677036\n"
     ]
    }
   ],
   "source": [
    "print(\"Accuracy score:\", accuracy_score(y_test, y_pred))\n",
    "print(\"Log loss:\", log_loss(y_test, y_pred_prob))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 326,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Submission\n",
    "#create_submission_file(y_pred_prob, 'VC_SGD_MNB')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 327,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV4AAAEqCAYAAABdmDjNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XecVNX5x/HPly5NQESkq6ggCChS\nVARUNGpiJJZooigoYu9G/RkU7N1YsBvFmtgAg6IoAiKCBURQFCuggCgofek8vz/OXRjG3Z1hd3Zm\ndvZ553VfO3POvTPPHeIzZ8499xyZGc4559KnQqYDcM658sYTr3POpZknXuecSzNPvM45l2aeeJ1z\nLs088TrnXJp54s1Bkh6WdE2m49hWkvaUNE3SCkkXluB1yuT5x5O0UtKumY7DpZ4n3gyTNFrS9QWU\nHyNpoaRKCY7vK2libJmZnW1mN6Q61uj9qkgaLOkbSaskzZH0hKQWKXj5K4DxZlbLzO4r7ouU1vlH\n523xXwqSLo7KByf5OuMl9U+0n5nVNLPvixmuy2KeeDNvKNBHkuLK+wDPmdmG9IdUpJeBPwN/B7YH\n2gNTgUNT8NrNgZkpeJ3S9DVwWlzZqVF5SiT6snU5wMx8y+AGbAcsA7rHlNUF1gDto+fbA08Di4C5\nwEDCl2braL+NwEpgabT/UODG6HFPYB5wGfAL8BPQL+a9dgBGAsuBj4EbgYmFxNoLWA00LeJ8GgH/\nA34DvgXOjKkbDLwYncsKQpLdL6obG53Hmuhc9gDGA/1jju+bHxsg4F/ROS0DZgBt488/en5mFMtv\nUWyNYuoMOBv4BlgCPACokHMbDDwLfAm0icraRM+fBQbH/Pu9Fv17LYkeN4nqboo7zyExcZwXxTE7\npqwlUAX4FLggKq8IvA9cm+n///pWvM1bvBlmZqsJyejUmOK/ArPMbHr0/H5C8t0V6BHt28/MviQk\njckWfpbWKeRtGkbHNwbOAB6QVDeqewBYFe1zGr9vzcXqBXxkZj8Wsc9/CIm+EXA8cLOk2Nbwn4H/\nAnUISXBI9DkcArwHnB+dS6IW5OFAd0KCrgOcCPwav5OkQ4BbCJ/pzoQvrv/G7fYnoBOh9f5X4A8J\n3vsZtvx7nUb4IolVAXiS0IJvRviyyj/Pf8ad5/kxx/UGugB7xb6Yma0DTgGul9QauIqQfG9KEKfL\nUp54s8NTwAmStouenxqVIakiIan8n5mtMLM5wF2ErohkrQeuN7P1ZjaK0NLaM3rt44BBZpZnZl/k\nv28hdiC0mAskqSnQDbjSzNaY2afA43GxTjSzUWa2kZDA2m/DecSfUy2gFaGF+qWZFRTbycATZvaJ\nma0F/g/YP65P+lYzW2pmPwDjgA4J3vtZ4G+SKgMnRc83M7NfzeyV6DNdQUiQPZI4p1vM7Lfoy3gr\nZvY54dfIcOByoE/0GboyyBNvFjCziYSfpcdEV7E7Ac9H1fUJPzXnxhwyl9B6TdavtnVfcR5QE9gR\nqATEtmCLas3+Smg1FqYR8FuUbAqLdWFcHNWK06dpZmMJrcgHgJ8lPSqpdiExzY05biXhPIqKqWaC\n9/6B0HVxM/BN/C8ASdUlPSJprqTlwASgTvRFV5SiPnsIX4otgFFm9k2CfV0W88SbPZ4mtHT7AG+Z\n2c9R+WJC6655zL7NgPnR45JML7cI2AA0iSlrWsT+Y4DOkpoUUr8AqCepVkxZbKzbahVQPeZ5w9hK\nM7vPzDoS+ln3AP5RSEybPztJNQgt9+LGlO9pQr95fDcDUfmeQBczq03oEoHQLw2F/5sl+rd8kNBf\n/AdJ3bYtXJdNPPFmj6cJfahnEvNzP/o5+SJwk6RakpoDl7Ll5+3PQBNJVbb1DaPXHgYMjlpprdi6\nrzl+/zHA28BwSR0lVYpiOlvS6VHLbxJwi6RqktoR+pSf29bYIp8Cx0axtYxeCwBJnSR1iX7ur2LL\nRcZ4zwP9JHWQVJXQSv0w6rIpiRcI/cwvFlBXi9Cvu1RSPWBQXP3PhP76pEnqA3QkXGC8EHhKUpEt\nc5e9PPFmiSgRTAJqEC46xbqAkFy+ByYSkskTUd1YwuiAhZIWF+OtzydceFtI6HP9D7C2iP2PB0YR\nEs8y4HNgP0JrGOBvhJ/DCwj9kYPM7O1ixAVh1MI6QqJ6iq0TeG3gMcKogbmE7oM741/AzN4BrgFe\nIfRP70boly0RM1ttZmMK6o8F7iGMVlkMfAC8GVd/L3C8pCWSEo5XltQses1TzWylmT0PTCF8Pq4M\nkplPhO62kHQb0NDMihrd4JwrAW/xlnOSWklqp6Az4ef88EzH5Vwu8ztkXC1C90Ijws0IdwGvZjQi\n53KcdzU451yaeVeDc86lWbntalCl7UxVaiXesRxq16qoobzlm/9ALNyPP8zlt18Xx0/2VCIVazc3\n21DQwJGt2epFo83siFS+d2kqv4m3Si2q7vnXTIeRlcZMuCfTIWSt9Rs3ZTqErHVEz/1T/pq2YXVS\n/52u+fSB+il/81JUbhOvc64sECj3ekRz74ycc7lDQIWKibdkXko6X9IUSWslDY2rqy7pQUmLJS2T\nNCGmTpJuk/RrtN0eO392dFfkVEl50d9Ekyx54nXOZTkp8ZacBYQZ3p4ooO5RoB5hjut6wCUxdQMI\nU3a2B9oRphE9K4SmKoThl88S5mF+Cng10S38nnidc1ks6mpItCXBzIaZ2Qji5m2WtCdhnugBZrbI\nzDaa2dSYXU4D7jKzeWY2nzDWvW9U15PQZXuPma21sGSVgEOKisUTr3Muu6WuxVuYLoT5Pq6Luho+\nk3RcTH0bYHrM8+lRWX7dDNv6hogZMfUF8sTrnMteItkWb/2o/zZ/G7AN79IEaEuY9KkRYeKop6LV\nPiDMz7wsZv9lQM2onze+Lr++yLGqPqrBOZfFlOzFs8Vmtl8x32Q1Yc7rG6MFA96VNI4w7eeXhBVb\nYifZrw2sNDOTFF+XX7+CIniL1zmX3Uq/q2FGgvqZbL1EVXu2rIY9E2gXt0p4OxKslu2J1zmXxVJ3\ncS2auL8aYaHQitFk/ZUISzP9APxftM+BhItmo6NDnwYuldRYUiPCCiNDo7rxhAn4L5RUVVL+4qVj\ni4rFE69zLnuJVLZ4BxK6Fa4irNq8GhhoZuuBY4CjCP2zjxEmnZ8VHfcIMBL4jDDx/+tRWf4K0L0J\nK7csBU4HekflhfI+XudcdkvRnWtmNhgYXEjdTKDAe56jEQtXRFtB9dMIyzIlzROvcy6LCSomd2da\nWeKJ1zmXvfKHk+UYT7zOuexW8lELWccTr3Mui+Xm7GSeeJ1z2c1bvM45l0ZK+s61MsUTr3Muu3lX\ng3POpZl3NTjnXDr5xTXnnEs/b/E651waSVAh99JU7p2Rcy63eIvXOefSzPt4nXMuzbzF65xzaSQf\n1eCcc2mnCrmXeHPvjLLEw4NOZu47tzDlpau3Kj/npB5MH34NU1/+JzdddMxWdU0b1mXR+3dxcZ9D\nN5dtX3M7nr/jDD4dNpBprwykS7td0hJ/ulx4Tn9a79KIgzp32Fx2+83Xs/cezel5QEd6HtCRt0e/\nsblu5uczOPKQbnTr1J7uXTqwZs2aTISdFpecN4C9Wzbh4P332Vw287MZHH1Ydw45YF9OPfEvrFi+\nHIAf585h14bb06tbJ3p168SVl5yXqbBTKixAoYRbWeOJt5Q8M/IDjjnvga3Kuu+3O3/quTed/noL\nHY+/iXuefmer+tsvP4633t96jbw7rzietyZ9QYdjb6Tzibcw6/uFpR57Op108mn8d/hrvys/+7yL\nGD9pKuMnTeWwPxwJwIYNGzi3/2ncce8DTPx4OiNGvUPlypXTHXLanPj3Pjz38sityi6/8GyuHnQj\nYyd9wpF/OoaH7rt7c13zXXZlzMSPGTPxY2771wPxL1c2KcktmZeSzo+Wfl8raWgh+wySZJJ6xZRV\nlfSEpOWSFkq6NO6YQyXNkpQnaZyk5oli8cRbSt7/5Dt+W5a3VdmAEw7iziffZt36DQAsWrJyc93R\nPdsxe95ivvhuS2KtVaMa3fbdjaHDJwOwfsNGlq1cnYbo0+eAbgdRt269pPYd987b7NV2b9ruHRZ8\nrbfDDlTMwdUJ8nU98CDq1q27Vdl3335N1wMPAqD7wYfy+sjhmQgtjRK3drehxbsAuBF4osB3knYD\njgd+iqsaDOwONAcOBq6QdER0TH1gGHANUA+YAryQKBBPvGnUsnkDDtxnNyY8fTlvPX4RHfdqBkD1\nalW4rN9h3PTIqK3236XxDixespJHrzuFyf+5kgev/TvVq1XJROhp9+9HH6RH13248Jz+LF2yBAhJ\nRxIn9D6KQ7p14v5/3ZnhKNNvz9ZtGD0qtIJfG/EKC+bP21z3w9w5HHZQZ449qhcfTpqYqRBTLlWJ\n18yGmdkI4NdCdhkCXAnEL1R5KnCDmS0xsy8Ji2H2jeqOBWaa2UtmtoaQpNtLalVULFmReCXNkbRa\n0sqYbUhMfc+o+X9F3HEtovL8Y+ZIuir9Z5CcShUrULd2dbqfeidX/2sEz95+OgDXnPNH7n92LKtW\nb/3vXalSRTq0aspjL73H/n+7jbzVa7n89MMyEXpa9e1/Fh/P+Ipxk6ayU8OdufbqfwCwccNGPpw8\niYcff5rX3nqXUSNHMGF8kato55y7hzzC0Mcf5g89urJy5UqqVA5fxA0a7szHn3/L2+99xOCbb+fc\nM0/b3P9b1lWoUCHhBtSPuhHytwHb8h6STgDWmdmouPK6QCNgekzxdKBN9LhNbJ2ZrQK+i6kvUDaN\najjazMYUUnca8Fv09/YC6uuY2QZJ+wPvSPrUzN4srUCLa/7PSxnxTvg3mjJzLps2GfXr1qRT2+b8\npVcHbrq4N9vX2o5Nm4w169YzfMw05v+ylI8/nwvA8DGfclm/3E+8DRrstPlxn75ncPIJvQFo1Lgx\n+x94EDvUrw9Arz8cyYxPp9G95yEZiTMTdt+jFf8dHnLDd99+zTtvhQuPVatWpWrVqgC067AvLVrs\nyvfffUP7fbZp8dvsk3wf7mIz269YbyHVBG4GDi+gumb0d1lM2TKgVkz9orhjYusLlBUt3qJIqk7o\ndzkP2F1SoR+umU0GZgJt0xTeNhk5fgY9O+8BQMtmDahSuRKLl6yk1xn30OqPg2j1x0EMeW48d/z7\nLR5+YQI//7qCeQuXsHvzBgD07Lxnzl1cK8jChVu62EaNHEGrvULj4eBDD+eLmZ+Rl5fHhg0bmDRx\nAnu0ap2pMDNi8aJfANi0aRP33nErffqdCcCvixexceNGAObO+Z7Z339LsxZlfwSMUtvHW5jrgGfM\nbHYBdfkXYmrHlNUGVsTU12ZrsfUFyqYWb2GOI5zcS8DJhP6WKfE7KXz6BxCa+NMKeqHo50f4CVK5\nZkG7pMxTt/TloI67U79OTb598wZueHgUT42YzCODT2bKS1ezbv1G+l/7TMLXufS2l3jy5r5UqVSR\nOfMXM2DQs6Uad7oN6HcK77/3Lr/9uph2e7bgiquvZdLEd/l8xnQk0bRZC+6870EA6tStyznnX8zh\nPfZHEr0OP4LDjzgqw2dQes45ow+TJ07gt18X03GvXbnsqmvIW7WSoY8/DMCRR/fmpFNOA+CD9ydy\nxy3XUaliJSpUrMitd9+f9EXLbJeG4WKHAk0knRs93xF4UdJtZnabpJ+A9sDbUX17QgOP6O9pMbHW\nAHaLqS+QzCyF8RePpDlAfWBDTPE/zOwxSWOAz83sYkl/A+4DGpnZekktgNmEpr0BC4GHzOy+RO9Z\noXoDq7rnX1N7Ijnix/fuyXQIWWv9xk2ZDiFrHdFzf6ZPm5rSLFlph12t9lE3JtxvybMnT03U1SCp\nEqGxOQhoApxJyDnbA7HjEj8GLgXeMLOVkm4F9gd6AzsB44B+ZvampB2Bb4HTgdcJreceZta1qFiy\nqauht5nVidkek9SUMHzjuWifV4FqwB/jjq1vZnXNrHUySdc5V0YIVEEJtyQNBFYDVwGnRI8Hmtmv\nZrYwfwM2AkvMLL+bYRDhgtlc4F3gjvxrSGa2iPCr/CZgCdAFOClRINne1dCH8OUwMubnRjVCd8OI\nTAXlnEufVHU1mNlgwnCvRPu1iHu+ltCiPb2Q/ccARQ4fi5ftifdUQtP94ZiyzsBLknbITEjOuXTJ\nv7iWa7Ip8Y6UtDHm+cdAC+CBqDmf73+SvgX+Bvz+XlPnXE7xxFtK4pv2SewfOzg59/5VnHNb5OB/\n4VmReJ1zrkAi/860nOKJ1zmX1byrwTnn0sgvrjnnXCbkXt71xOucy2LyrgbnnEs7v7jmnHPplnsN\nXk+8zrns5l0NzjmXRmV1FeFEPPE657KaJ17nnEuzbZj2sczwxOucy2re4nXOuXTycbzOOZdeAnIw\n72bV0j/OORcndasMSzpf0hRJayUNjSnvKultSb9JWiTpJUk7x9RL0m2Sfo222xXzppI6SJoqKS/6\n2yFRLJ54nXNZrUIFJdyStAC4EXgirrwu8Chh4YXmhKXZn4ypH0BY6LI90A74E3AWgKQqhLUgn41e\n5yng1ai88HNKNmLnnEs7ha6GRFsyzGyYmY0Afo0rf8PMXjKz5WaWBwwBDozZ5TTgLjObZ2bzgbuA\nvlFdT0KX7T1mtjZabFfAIUXF4onXOZe1RNIt3vpRN0L+NqAEb9sdmBnzvA0wPeb59Kgsv26GmVlM\n/YyY+gL5xTXnXFZLskW72Mz2K/l7qR1wLXBMTHFNYFnM82VAzaifN74uv75WUe/jidc5l9XSNZxM\nUkvgDeAiM3svpmolUDvmeW1gpZmZpPi6/PoVRb2XdzU457KWlNKLa0W8j5oDY4AbzOyZuOqZhAtr\n+dqzpStiJtBOW387tGPrrorf8cTrnMtiKR1OVklSNaAiUFFStaisMTAWeMDMHi7g0KeBSyU1ltQI\nuAwYGtWNBzYCF0qqKun8qHxsUbF4V4NzLqulsKdhIDAo5vkpwHWAAbsCgyRtrjezmtHDR6L6z6Ln\nj0dlmNk6Sb2jsluBL4HeZrauqEA88Trnslqq+njNbDAwuJDq64o4zoAroq2g+mlAx22JxROvcy57\nbcM43bLEE69zLmvlj+PNNZ54nXNZzWcnc865NMvBvFt+E2+7Vk155717Mh1GVup56/hMh5C1Jg8s\n8hb8cq1iaXQJlLf5eCVVT/ZFooklnHMupXJ1Pt6iWrwrCePbklExBbE451yc1NyZlm2KSrxHpi0K\n55wrRLnqajCz0ekMxDnnfidHx/EmPVeDpB2ipTP+JWmHqKyTpKalF55zrjwLfbypmashmyQ1qkFS\ne+AdYBGwG3A/YRb3PxOWy+hTSvE558q5sphYE0m2xXsX8LiZtQbWxpS/ARyU8qiccy6Sjmkh0y3Z\ncbydiBZ3izMfaJi6cJxzLkaO9vEmm3jX8vtZ1gH2ABanLhznnNtClM0+3ESS7WoYCQyUlJ+oLZo8\n+FZgeKlE5pxzpG6V4WySbOK9HGgCLAS2I8yu/j2wDri6dEJzzjmoICXcypqkuhrMbImk/YEjgH0J\nCfsTYJSZbSrF+Jxz5Vj+mmu5JulxvGa2ycxGmdmNZna9mb3mSdc5V9oqKPGWjOg+hCmS1koaGld3\nqKRZkvIkjYsWv8yvqyrpCUnLJS2UdGmyxxZ6TsmFDJKOkPSWpHnR9rYkv63YOVeqUngDxQLgRuCJ\nuNevDwwDrgHqAVOAF2J2GQzsDjQHDgaukHREkscWKKnEK+kC4H/Az8DN0bYQGCHpwmRewznniiNV\nF9fMbJiZjSDc/BXrWGCmmb1kZmsIiba9pFZR/amEZd+XmNmXwGNA3ySPLVCyw8muAi4xswdiyh6U\nNJmwcud9Sb6Oc84lTYQhZUmoL2lKzPNHzezRJN+mDTA9/4mZrZL0HdBG0s9Ao9j66HHvRMcCswp7\nw2QT7/ZAQZPmvAXcnuRrOOfctpGSnWB9sZntV8x3qUmYDiHWMqBWVJf/PL4u0bGF2pZxvMcUUH4M\nMCrJ13DOuW2WhnG8K/n9DWK1gRVRHXH1+XWJji1UUStQnBvz9DPCDRQHAR9EZV2BHniL1zlXSgTp\nGKc7Ezht83tKNQiTgc2MhtL+BLQH3o52aR8dU+SxRb1hUV0N18Q9XwN0ibbYsguAW4p6E+ecK65U\n5d3ozttKhBVzKkqqBmwg3H17h6TjgNeBa4EZZpbfR/s0oeE5BdgJOBPoF9UlOrZARU2EvnMxz885\n51ImhXM1DAQGxTw/BbjOzAZHiXMI8CzwIXBSzH6DgIeAucBq4DYzexPAzBYlOLZA5XaVYedc9pNS\nt3qxmQ0mDPcqqG4MUOAQMDNbC5webdt0bGGSTrySWhDGrDUDqsS98bkFHOKccyWWezcMJ78CxWGE\nGyhmsWXc2q6EvpKPSi0651y5V56nhbwZuNXM9iHMzXsioeX7LiEhO+dcyoVRDamZqyGbJJt4WxE6\njiFcBdzOzFYRruD9ozQCc845kpinoSy2iJNNvKvY0q/7E2GcGoABO6Q6KOecy1ee11z7CDgA+BJ4\nkzBurTVwHN7H65wrJfldDbkm2RbvP4BPo8eDgEnAGcAv0V9XhAvP6U+rFo3o1qnD7+qG3Hs39WtW\n5tfFYem6+++5i577d6Tn/h3p1qkDDWpXZclvv6U75FJ1fe+9GH9ld4ad33Vz2eFtGjD8gq5Mv+5Q\n9mq05Tb3P7ZryEvndtm8Tb/uUPZsGG6fP3LvnRh2fldeOa8LD53agTrVK6f9XNJpyH330Gmfvem8\nbzv69fk7a9as4d1xY+nWdT8679uOAWf0ZcOGDZkOM+XKbVeDmX1lZlOjxyvMrJ+Z7QH8hdwc7ZFS\nJ518Gi+MeO135fPn/ci7Y8fQpGmzzWUXXHwZ4ydPZfzkqQy87kYO6NaduvXqpTPcUvfqtAWc8/S0\nrcq++WUll/xnBlPnLt2q/PUZCznhwQ854cEPufqVz1mwdDVfLVxJxQriyqP25PQnpnLcAx/y9cKV\n/K1L03SeRlotmD+fhx+4nwmTPuKjT2awcdNGXvzv85zVvx9PPvM8H30yg2bNmvPcM09lOtSUUxJb\nWZP0ROiF2Av4JhWB5LIDuh1E3bq/T54Dr7ycQTfeUug39rCXXuDYE04s7fDSburcpSxbvX6rstmL\n8pizOK/I447cuyGjPvsZiP6DE2xXuSIANatWYtGKtaUSb7bYsGEDq1evZsOGDeTl5VGjRg2qVq3K\n7rvvAcDBh/bifyOGZTjK1JJyc821kiZeV0xvvD6SnRs1ou3e7Qusz8vLY+yY0Rx9zLFpjix7HbH3\nTrwxYyEAGzYZN46cxbDzuzL2ioPYrUENhk2dn+EIS0+jxo258JLL2Gv3FrRs0Zjta2/Pscf/lfXr\n1/PJ1DAN7avDX2HevHkZjjT1cvHiWlYkXklzJPWKK+sraWJM/WpJKyX9LOlJSTWjuvGS+mci7uLK\ny8vjX3fcwlUDBxe6z+hRr9G56wE5181QXHs3qc2a9Zv49pdVAFSqIP7aqQknPPghh9z+Hl8vXEn/\n7rtkOMrSs2TJEl4f+T8+m/Ud38yex6q8Vbzwn+d48pnnueofl9GzW1dq1qxFpUq5NwtAeV7ePRsc\nbWY1CascdyJMeFEmzfn+O36YM4ce+3dkn71asmD+PA7p1pmff164eZ/hL7+Yk90MxXXk3g0ZNWPL\n57PnzuEC3LwlqwEY/fnPdGi2fUZiS4fxY8fQvEULdtxxRypXrsyfj/kLH34wmS5d9+etse8yfuIH\nHNjtIHZr2TLToaaUSNzNUBa7Gor8epS0V4Ljd0tQn3JmNl/SG0DbdL93quzVdm9mzVmw+fk+e7Vk\nzIQP2KF+fQCWL1vGpPcn8NC/c+9CSXFIYdRD339P3Vz2y/I17NagBnWrV2ZJ3nr2b1mP7xetymCU\npatJ02Z8/NGH5OXlsd122zF+3Fj27bgfi375hR0bNGDt2rX86647+MeV/5fpUFOrjLZoE0n0u+Rz\nwk0ShVGC+pST1BQ4irCy57YeOwAYAGw1kqC0ndn3FN5/711++3Uxe+/Rgiv/eS2nnFbgREcAvD5y\nBD0POYwaNWqkLcZ0uu2EtnTapS51qldmzOXdeGDs9yxbvZ6r/7gndWtU4cE+HZj100rOjkY+dGxe\nl4XL125u3QIsWrGOh8Z9z9D++7Fh4yYWLF3DwGFfZOqUSl2nzl3o/Zfj6NZ1PypVqkT79h3od8aZ\nXD/4Gt4c9TqbNm2i/4Cz6XHwIZkONeXK4nCxRGRWeN6UtGcyL2JmX5UoCGkOUJ9wO3K+KsAnZtYt\nrn4ZYcLhy8xstaTxwLNm9vi2vGeHfTvaO+99WJKwc1bPW8dnOoSsNXlg7iW2VOl+QGc+mTolpVly\np5Zt7cQ7X0643/1/aT21BGuupV2RLd6SJtRt1Dua1xIIF9eA/oXVO+fKhzI4aCGh3LsE6pzLKbmY\neMvSqIaiVJJULWbL7XtHnSsnwnCx1NwyLKmFpFGSlkhaKGlItA4bkjpImiopL/rbIeY4SbpN0q/R\ndrtK2PGcK4n3IcJaSPnbk5kNxzmXKimcj/dBwvwyOwMdCKuknyupCvAqYerbusBTwKtROYQL8r0J\nqwu3A/4EnFWicyrJwaliZi3i+2/NbKiZdSusPma/nmamuO2UdMTtnCtdIqy5lmhL0i7Ai2a2xswW\nEmZabAP0JHS73mNma83svuit86+kngbcZWbzzGw+cBfQtyTntU2JV1JNSe39p7xzLl0qJLEB9SVN\nidkGFPBS9wInSaouqTFwJFuS7wzbeojXjKgctix3lm96TF2xzykhSTUkPQ0sB6YCTaPyIZL+WZIA\nnHOuKEneMrzYzPaL2R4t4KXeJSTM5cA8YAowAqhJGKYaaxmQPz9pfP0yoGZJ+nmTbfHeQlj+5wBg\nTUz5W8AJxX1z55wripK4XTiZW4YlVQBGE268qkG4L6AucBuwEqgdd0htYEX0OL6+NrAyroW8TZJN\nvMcAF5jZB2x9p9oXhNWGnXOuVKRokpx6hF/qQ6J+3F8JF+GPAmYC7eJasO2icqK/sdMIto+pK5Zk\nE++OhKuB8XLznlbnXFYQYSa6RFsiZrYYmA2cI6mSpDqEi2bTgfHARuBCSVUlnR8dNjb6+zRwqaTG\nkhoBlwFDS3JeySbeqYRvhs1Jdyw3AAAbbElEQVTnEf09HZhckgCcc64oKZwW8ljgCGAR8C1hCoJL\nzGwdYbjYqcBSQl7rHZUDPAKMBD4jzF/zelRWbMneufZPYJSkVtEx50nKH4bRoyQBOOdcobZtnG6R\nzOxTQs4qqG4a0LGQOgOuiLaUSHbNtQmEBNsAmE/45lgFHGhmvsqwc67UKIn/lTVJz9UQLXbpM3M7\n59ImV5d3TyrxSqpeVL2ZFb1KoXPOFdM23JlWZiTb4l1J0ROeV0xBLM45t5Vy3eIl3FoXqzKwD2G+\n3GtSGpFzzuUrp0v/AGBmowsofk3S18AphHFuzjmXcmVxMctESjo72RS2zODjnHMpld/VkKJpIbNG\nsVegiOaqPI8wvMw550qBqJiDLd5kRzUsYuuLawLqAOsId3s451zKiXLcxwsMjHu+iXDb3SQzK2gO\nB+ecK7ky2pWQSMLEG61JtB4YFc3a7pxzaVMuL66Z2QZgCFC19MNxzrkt8rsaUjRJTtZItqvhI8Ic\nlHNLMRbnnPud8nzn2hDgrmguyqmECXI2M7MvUh2Yc86JLFmRN8WSTbwvRn8fjP7mj3BQ9NhvGXbO\npZ7C8j+5JtnE27pUo3DOuULkXtpNkHglPQFcZGZfpSke55zbLNy5lnupN1H3yWnAdukIxDnnCpLK\nW4YlnSTpS0mrJH0n6aCo/FBJsyTlSRonqXnMMVUlPSFpuaSFki4t8TklirOkb+Ccc8UnpMRbUq8k\nHUZYzr0fUAvoDnwvqT5h2fdrCKsRTwFeiDl0MLA70Bw4GLhC0hElOatk+niLvXa8c86VRIpHNVwH\nXG9mH0TP5wNIGgDMNLOXoueDgcWSWpnZLMK0CP3MbAmwRNJjQF/gzeIGksw5LZS0saituG/unHOJ\nJNnirS9pSsw2IO41KgL7ATtK+lbSPElDJG0HtCEs8w6Ama0CvgPaSKoLNIqtjx63Kck5JdPiHUBY\n8tg559Iuyf7OxWa2XxH1OxEWcDgeOIgwDcKrhHloahLmnom1jNAdUTPmeXxdsSWTeEfm6kQ45p0o\nBfp4UK9Mh5C19rxsZKZDyFoL5y1LvNM2kkjVtJCro7/3m9lP4bV1NyHxTgBqx+1fG1hBWPYs//ma\nuLpiS9TV4KnJOZdRqbi4FvXPzqPgnDaTMCVC/vvVAHYj9PsuAX6KrY8ezyzBKfmoBudcdlMSW5Ke\nBC6Q1CDqu70YeA0YDrSVdJykasC1wIzowhqEpc0GSqorqRVwJjC0JOdUZFeDmeXibdLOuTIkhfdP\n3ADUB74mdBu8CNxkZmskHUeYk+ZZ4EPgpJjjBgEPESYJWw3cZmbFHtEAJVj6xznnSlsYTpaazGtm\n64Fzoy2+bgzQqpDj1gKnR1tKeOJ1zmUx5eQtw554nXNZLQfzride51z2SmVXQzbxxOucy15ldGmf\nRDzxOueymide55xLI5GyO9eyiide51xWk/fxOudceuVgg9cTr3Muu3mL1znn0iisuZbpKFLPE69z\nLnvJ71xzzrm0y72064nXOZfFcnV5d0+8zrmslntp1xOvcy7b5WDm9cTrnMtq3tXgnHNplntpN/Ga\na845l1kpXHQNQNLuktZIejam7O+S5kpaJWmEpHoxdfUkDY/q5kr6e0lPyROvcy5rhbya+H/b6AHg\n483vIbUBHgH6ADsBecCDcfuvi+pOBh6Kjik272pwzmWvFM/HK+kkYCkwCWgZFZ8MjDSzCdE+1wBf\nSqoFbAKOA9qa2UpgoqT/EZL0VcWNw1u8zrmsJiXegPqSpsRsA37/OqoNXA9cFlfVBpie/8TMviO0\ncPeIto1m9nXM/tOjY4rNW7zOuSyWdFfCYjPbL8E+NwD/NrMftXUzuiawLG7fZUAtYGMRdcXmidc5\nl9VS0dUgqQPQC9ingOqVQO24strACkJXQ2F1xeZdDWlw4Tn9ab1LIw7q3GFz2e03X8/eezSn5wEd\n6XlAR94e/QYA69at44Kzz6B7lw703H9f3n/v3UyFnRFLly7lbyceT/u2reiwd2s+mDyZV15+iX3b\nt6F6lQpMnTIl0yGWqjv+1p6pNx7OW1f12Fx29Z9b887VB/PmlT145Iz9qL3dlvZSq0a1GH7xgbx9\nVU9GX9mDqpW2/k/68f6dtnqtsiaZAQ1J5uWeQAvgB0kLgcuB4yR9AswE2m9+T2lXoCrwdbRVkrR7\nzGu1j44pNm/xpsFJJ5/GGWedy/kDTt+q/OzzLuK8iy7dquyZoY8DMOHDT1m06BdOOvZPvP3uB1So\nUD6+Iy+/5CIOP/wI/vPCy6xbt468vDzq1KnDf18cxvnnnpXp8ErdSx/9yFPvzeHuU7Z8Sb/31WJu\ne20WGzcZVx3dmnN77c6tI7+kYgVxT599ueSZaXy5YDl1qldm/cZNm487ol1D8tZtyMRppFZqLq49\nCvw35vnlhER8DtAAmCzpIOATQj/wMDNbASBpGHC9pP5AB+AY4ICSBFM+/mvOsAO6HUTduvUS7wh8\nNetLuvc8BIAdd2zA9tvX4dNPcruVl2/58uVMnDiBvqefAUCVKlWoU6cOrVq3Zo8998xwdOnx0Xe/\nsTRv3VZl7321iI2bDIBpc5ewc51qAHRvtSOzFiznywXLAViat55oN6pXqUj/g3fj/tHfpC/4UlIh\nmhqyqC0RM8szs4X5G6F7YY2ZLTKzmcDZwHPAL4T+23NjDj8X2C6q+w9wTnRM8c+pJAe7kvn3ow/S\no+s+XHhOf5YuWQJA27bteOP1kWzYsIG5c2Yz/dNPmD9/XoYjTY/Z339P/fo7MuCMfnTdbx/OGdCf\nVatWZTqsrPLXLk0Z/+UvAOyyYw3M4Omzu/D65d0565DdNu932R9b8di471i9fmOmQk2ZFN8/AYCZ\nDTazU2KeP29mzcyshpkdY2a/xdT9Zma9o7pmZvZ8iU6INCZeSXMkrZNUP678U0kmaX9JyyVVjKl7\nrJCyh6PHbSS9JWmJpKWSpko6Kl3nVBJ9+5/FxzO+YtykqezUcGeuvfofAPz91H40atyYXt27MPDK\ny+jUZX8qVSwfPUIbNmzg02mfcOZZ5/DBlGlUr1GDO2+/NdNhZY3zD9udDZuM4VPmA1Cpgui0az0u\nemYax937Pke0a8iBe9Rnr8a1aVG/OqNnLMxwxCmQwk7ebJLuFu9s4G/5TyTtTWjCAywGKgL7xux/\nELAgrqw7MCF6PBJ4m3BHSQPgQmB5aQSeag0a7ETFihWpUKECffqewbSpoTuhUqVK3HjrXYyfNJVn\nXhjG8qVL2bVlywSvlhsaN2lC4yZN6NylCwB/Oe54Pp32SYajyg7HdWrCoW0acNHT0zaX/bR0DR98\n+ytLVq1jzfqNjPviF9o22Z59W9Rl76Z1mHjtobx80YHssmNN/nv+/hmMvmRK4c61jEt34n0GODXm\n+WnA09Hj9cAHhMSKpAZAFeCFuLI9gAlRy3kX4DEzWxdt75vZxLScSQktXPjT5sejRo6g1V5hPHZe\nXt7mn9fjx46hYqVK7Nlqr4zEmG4NGzakSZOmfP3VVwCMH/sOrVqXj3MvSo9WO3JOr5ac8djHrInp\nOnh31iJaN6pNtcoVqVhBdGm5A98sXMGz78+l87Vv0+36dzj+3veZvWglJw2ZnMEzKD6R9A0UZUq6\nf8N+APSR1JowTONEoBtwY1Q/gZBk74r+Toy282PKZpvZPIUR0N8Cz0p6HJhsZj8X9ebR3SwDAJo0\nbZbiUyvcgH6n8P577/Lbr4tpt2cLrrj6WiZNfJfPZ0xHEk2bteDO+8Kt4YsX/cJfe/+RChUqsHOj\nRjz42NC0xZkN7r7nfvqdejLr1q2jxa678ujjT/LqiOFcevEFLF60iGOP+SPt2ndg5KjRmQ61VNx3\n6r7s33IH6taswgfX9eJfb3zFub12p0qlCjx7blcgXGD754ufsXz1eh4f/x0jLzsIwxj3xS+M/eKX\nDJ9B6pXFxJqIzCw9byTNAfoDXYEawLuEW/eOJLR2dyEM73gFqA/cA3xOuIr4HdAwKqttZv2i12xC\nuF/6yOj4icAZZpbwUm6HfTvamAkfpuz8cknNauWjT7k49rxsZKZDyFoL/3spa3/+JqVpsm37fe3l\nNxP/iG3dqMbUJO5cyxqZGNXwDPB3oC9buhnyfUC4fa8toXX7XjQxxY8xZfn9u5jZPDM738x2A5oD\nqwp4TedcGZaLXQ1pT7xmNpdwke0oYFhc3RrCdG1/AnY2s1lR1XtRWTtiEm/csT8Spm9rWzqRO+cy\nIQcHNWRsHO8ZwCFmVtAgzQnAxYRp2/JNjMoWRjMHIamupOsktZRUIbrYdjqh1eycyxU5mHkzknjN\n7DszK+x2rHcJQ8NiO3YmRmWxrd11hD7hMYQhZJ8DawldGM65HCCl5s61bJO2qyhm1qKQ8g3EfGeZ\n2WjivsOi0QrxZasIw9Gcczms7KXVxPzytXMuu+Vg5vXE65zLYmXzzrREPPE657JaGezCTcgTr3Mu\na+XfMpxrPPE657KadzU451yaeYvXOefSLAfzrq9A4ZzLYgJJCbeELyNVlfRvSXMlrZA0TdKRMfWH\nSpolKU/SOEnN4459IlqUYaGkSwt+l+R54nXOZa0UzsdbiTDZVg9ge+Aa4EVJLaLpBoZFZfWAKYR5\nwPMNBnYnTMR1MHCFpCNKcl7e1eCcy2qp6GqI7nQdHFP0mqTZQEdgB2Cmmb0EIGkwsFhSq2iirlOB\nfma2BFgi6THC1ARvFjceb/E657Jaki3e+pKmxGwDin5N7URYzWYm0AaYnl8XJenvgDaS6gKNYuuj\nx21Kck7e4nXOZbUkh5MtTnYidEmVCUu5P2VmsyTVBBbF7baMsMx7zZjn8XXF5onXOZfVUjmcTFIF\nwmIM6whLigGsBGrH7VobWBHV5T9fE1dXbN7V4JzLWsl0MySbmKN1Gv9NWJX8ODNbH1XNBNrH7FcD\n2I3Q77sE+Cm2Pno8syTn5YnXOZfVUri8+0NAa+BoM1sdUz4caCvpOEnVgGuBGTEr4DwNDIwWX2gF\nnAkMLck5eeJ1zmW3FKxAEY3LPQvoACyUtDLaTjazRcBxwE3AEqALcFLM4YMIF9vmEhZquMPMij2i\nAbyP1zmX5VI0nGxuUS9lZmOAVoXUrSUsK3Z6CkIBPPE657Ja2VzaJxFPvM65rJWr00J6H69zzqWZ\nt3idc1ktF1u8nnidc1nNJ0J3zrk0kqBC7uVdT7zOuSznidc559LLuxqccy7N/OKac86lWQ7mXU+8\nzrnslsyaamWNJ17nXNbK1TvXZGaZjiEjJC0izDaUDeoDizMdRJbyz6Zo2fT5NDezHVP5gpLeJJxj\nIovNrEQLUKZTuU282UTSlGSXLSlv/LMpmn8+ZZPP1eCcc2nmidc559LME292eDTTAWQx/2yK5p9P\nGeR9vM45l2be4nXOuTTzxOucc2nmidc559LME28WUi7eI+mc28wTbxYyv+LpiknS0ZIKXKbcZQ+f\nqyGLSDoTaAh8CEwxs98yHJIrQyS9AOwI3CRptpmtzXRMrmA+nCxLSBoBNAZWAg2Ae83sUUnyFrBL\nJEq6jYA/AqvNbH1cfQUz25SR4NzveFdDFpA0EtjOzDqZ2cHAu8BZkiqV96Qr6XhJbTIdRzaTtB/Q\nDDjCzJab2XpJO0rqIekwADPbJMn/e88S3tWQYZIOJrRSesYUv0po/dYAlmUgrKwgqS1wA/A/Sf82\ns68zHVOWMqCWma2S1ADYF3gaWALUlTTBzI73Fm/28G/ADDOzccAgYLSkDpIqA48BY82s3CZdADP7\nHDgV6AqcIWmP/DpJFWMeb5+B8DJO0iWSTjezqUA1STOAJ4F/AfcDPYC/A53zW74uO3iLN4Py+93M\n7IboZ+AUYDVwtZndH+1T7vp4Jd0M7AYsAu4BLiIkEpP0pJl9ZWYbo30vAtpLOtvM1mUs6DSTVAPo\nQmg8PUFo5d4ITAdmm9nYaL88wrzTKzIUqiuAX1zLAEkdga/NbEVc+QXAvUBbM/tCUuX4iyS5TtKr\nhImv3wE6EBJLb0LrbRAwGXjCzL6SdDlwO9DRzKZlKOSMkbQbMBW4w8xuKmSfs4CLgV5mNj+d8bnC\neYs3zSTdCFwNTJR0EzDDzH4CMLP7Je0IfCrpD2Y2rjxdjZb0MtA4f2JvSb2BW4GdzewdSQYMBpZG\nfZlnA53KU9KVdDbwOFDBzL6TdA4wWNLk/FZutF834EDgH8DhnnSziyfe9PscmA38BgwFpkkaCwwB\n1prZtZI2Au9I6m5mEzMXavpIqgf8AbhXUnUzywO+APKAmlGXy9jopr77CBcfu0X9m+WCpGOBBwl9\n3mMkTQBeInxuR0qaambLom6II6P9DjazzzIWtCuQdzWkiaRqZrZG0q7AOOBy4BPgFKAvoT/zDcLP\nxpWSzgfeMbMvMxVzuki6ltCFMA/4H+EL6Q1gBDDEzG6P238/whpbc9IbaWZJakr4TFYREu6RhBZt\nE8IXd18zez/adzvCEEW/CScL+aiGNJD0GHAygJl9D9xJ6JvcZGbXEZJwR+B44CdJlwIPlZOkWxfY\nDrgJqAccA5wOTAKey0+6sWNQzWxKeUu6AGb2I3ABUBH4HngRGAlUARYAD0iqFe272pNu9vLEW8qi\ni0X7mNm/Y4rHAT8DzSR1Jwz/ORfYm9D/OyL/qn2uM7MlwAOEsct3A3WBw4H5hL7cetF+5aKfO56k\nayRNltQy6oKZBLwG7GVmQ4HzgV6ElYbbEf5/5LKcdzWUIknDgR3NrFtMWRUzWyfpQaAP4Rbhf5rZ\nE5mKMxOiu/I2xDyvDNwM/AnoBywntOb+Q7h9elFGAs0gSVcS7mK8CagMjAEeIszncStwiZl9LakZ\nsDthOFk/M5uVoZBdkjzxlhJJjxMuerSIGXO6K+HC0PnAWuBtYIKZnVuexutKuoVwkfFlYJ2ZmaRG\nhD7vz4FawGWEVtxkQkv4pvLU6o36+O8D3gIGAEcDhwKtgZMIN5YcAPTIH79cnkbAlHXe1VB6niFc\nkT8cQFITYALwadQ/uRSYRmi9lLepIMcB1wJHR0m3KWFGtnsJX1YjCH3gOxNuEvhPOUworwPvAZ0J\nXQuvE4bPvU5Ixr8RumXOzp+/uRx+RmWWJ95SYmbvEloqd0djLycDD5rZwKh1u5qQbA6WVKc8TX5u\nZm8BZwI3RJ/NJOBhM7sl+nXwPDAWGAj8aGbfZi7a9Ituh15ASLKnEb6gJwA1zOxywi+m1sAewLFA\ntQyF6orJuxpKWXTxbBTwmpmdFFN+CeHn4klm9lWm4sukgj6b/J/LUdfDWjP7NaNBppGk/YGPYrqm\n+gGXEFq9jxAuonUzs9mSGhPG6X5eXv//U5Z54k0DSQcS7jb6p5kNi1p5txBu4yw3NwAUJPpsHgP+\nCYyObpwodyQNJXwRP04Y8TIo+gK6FfjJzO5VmLO5HeFOtHL1KyDXeOJNE0k9CBO9fEqYBvIwM/sk\ns1Flh+izeZBwVf6V8jTZTT5JBxD6bp8GdgEEjCb0c1cxs4uj/d4mXBfYJ3ZUiCtbPPGmUTT37vOE\nCaunZzqebCKpF+FXwCHxkweVF1Hr/2HgUsLE5rUI/dz1gD+Y2dvRfk3MbF7GAnUl5ok3zfJvHc50\nHNkoZo6Gciv6cr4PuNbMhkvqALQk3EK92kcu5AZPvM5lGUk9Cf3e/2dmL2c4HFcKfHYy57KMmY2X\n1B94KLqjr1z2e+cyb/E6l6W83zt3eeJ1Lot5v3du8sTrnHNp5rcMO+dcmnnidc65NPPE65xzaeaJ\n1znn0swTr3POpZknXldskj6XNDjm+RxJl2cgjv0kmaQWpfgegyV9noLXychn5LKLJ94cImlolIBM\n0npJ30u6U1KNNIXQiTDLWEKS+kpaWcrxxL7feElD0vV+zhXFbxnOPWMIi2hWBg4izO9aAzinoJ0l\nVTaz9al44/K4IKVzxeEt3tyz1swWmtmPZvY88BzQG8LkK1Fr+ChJH0laR1jjDElHS5oqaY2k2ZJu\nklQl/0UlNZD0qqTVkuZKOj3+jeN/RkuqLekhST9Fr/ulpBOjSWCeBGrEtNAHR8dUkXSbpHmSVkn6\nWNIf4t7nCEmzotd8j7AETolIulXSV9H5zZF0u6TfLakjqb+kH6L9RkiqH1ffT9IXUWxfS7pEkv93\n5rbiLd7ct5rQ+o11G2EV32+BFVFiew64iLC2VzPCvLBVgfxEOhRoTlh+Jg/4F9CisDeN1pB7g7Ag\nYz/ga2BPwvpgk4CLCcu57xYdkt/t8GRU9ndgHnAUMFJSJzObHi2MOYIwe9cDhBUZ7k7+4yjUKuB0\nYD6wF+H81wLXxOzTAjgFOAaoDjwKPAH8OTrnM4HrgQuAqUDbKM71gHdzuC3MzLcc2QjJ8bWY550J\nS6S/ED3vCRhwXNxxE4Br4sp6E5KhCC1KAw6MqW8ObAQGx5TNAS6PHh8GbAJaFxJrX2BlXNlu0THN\n4spHEBYKhZCsvya63T0qGxjF16KIz2Y8MGQbPsuzgW9jng+OzrdZTFm36H13j57/APSJe52LgS8K\n+ox8K7+bt3hzzxHRRatKhJbuq4QWWKwpcc87Ap0lXRlTVgHYjrDMTGtCQvwov9LM5kpaUEQc+xDW\nCvtyG2Lfl5Dov4hbdLkqYdVholg+MLPYSUYmb8N7FEjS8YQk2RKoCVSMtljzzeyHmOcfEn25SFoK\nNAUekfRQzD6VCOfk3GaeeHPPBMKy8uuBBVbwhbNVcc8rANcBLxWw7yKKlziKc0wFQguyEyH+WKtL\n8LpFktQV+C/hM7gEWEroPrhzG14mvx83f7l65wrliTf35Nm2r0D7CdCqsOMkfUlILJ2IkoqkZkCj\nBK+5s6TWhbR61/H7FuU0QmJtaGbjCnndL4DjJCmm1du1iDiScSChNXtDfoGk5gXs11hSUzP7MXre\nmfC5fGlmP0uaD+xmZk+XMB6X4zzxOggXhF6TNBd4EdhAuDDU2cyuMLOvJL1J+Bk9gND6vJstrdCC\nvEP4Kf6KpEsI/bItgRpmNoLQ11lN0mGEhJtnZl9Leg4YKukyQvKuR+ib/t7MhhEuel0G3CPpQWBv\nQiszGfWjNcxi/RLF1ljSyYRuiz8Afyvg+NXAU5IuJXTDPAy8bmbfRPWDgfujbodRhK6efYHGZnZL\nkjG6csCHuTjMbDRhyfmDCf24HwFXES4W5esLzCb0tY4krJY8p4jX3AQcCbwPPAt8CdwLVInqJxES\n138I3RlXRIf2I4xsuB2YBbwGdAfmRsf9ABwLHAFMJ3QNXJXkqZ5ISPKx26VmNhK4A7gHmEG4MHht\nAcfPIXRJjIw+h++jePPP+XHCyIg+UWzvEbp9ZicZnysnfCJ055xLM2/xOudcmnnidc65NPPE65xz\naeaJ1znn0swTr3POpZknXuecSzNPvM45l2aeeJ1zLs3+H9p/SVqaUBgCAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2a3823927f0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Confusion Matrix\n",
    "conf_mx = confusion_matrix(y_test, y_pred)\n",
    "plot_conf_matrix(conf_mx, train.author.unique(), title=\"Voting Confusion Matrix\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prepare submission file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 238,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_submission_file(y_pred_prob, name):\n",
    "    result = pd.DataFrame(y_pred_prob, columns=['EAP', 'HPL', 'MWS'])\n",
    "    result.insert(0, 'id', X_ID)\n",
    "    result.to_csv(name+'.csv', index=False, float_format='%.20f')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train and test using grid searched params"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### MNB"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 328,
   "metadata": {},
   "outputs": [],
   "source": [
    "# MNB\n",
    "# clf__alpha: 0.01\n",
    "# tfidf__norm: 'l2'\n",
    "# tfidf__use_idf: False\n",
    "# vect__max_df: 0.5\n",
    "# vect__max_features: None\n",
    "# vect__ngram_range: (1, 2)\n",
    "\n",
    "mnb_clf = Pipeline([\n",
    "    ('vect', CountVectorizer(stop_words='english',max_df=0.5, max_features=None, ngram_range=(1, 2))),\n",
    "    ('tfidf', TfidfTransformer(norm='l2', use_idf=False)),\n",
    "    ('clf', MultinomialNB(alpha=.01)),\n",
    "])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 329,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Pipeline(memory=None,\n",
       "     steps=[('vect', CountVectorizer(analyzer='word', binary=False, decode_error='strict',\n",
       "        dtype=<class 'numpy.int64'>, encoding='utf-8', input='content',\n",
       "        lowercase=True, max_df=0.5, max_features=None, min_df=1,\n",
       "        ngram_range=(1, 2), preprocessor=None, stop_words='english',\n",
       "        ...lse,\n",
       "         use_idf=False)), ('clf', MultinomialNB(alpha=0.01, class_prior=None, fit_prior=True))])"
      ]
     },
     "execution_count": 329,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mnb_clf.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 330,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = mnb_clf.predict(X_test)\n",
    "y_pred_prob = mnb_clf.predict_proba(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 331,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy score: 0.835137895812\n",
      "Log loss: 0.406560686329\n"
     ]
    }
   ],
   "source": [
    "#create_submission_file(y_pred_prob, 'MNB_GS')\n",
    "print(\"Accuracy score:\", accuracy_score(y_test, y_pred))\n",
    "print(\"Log loss:\", log_loss(y_test, y_pred_prob))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 332,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV4AAAEqCAYAAABdmDjNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xd8VFX6x/HPNwSQuiABFaSoyIIo\noGJF7A3Liqv7kxULuIplLWtdVxGwd9eCip1V17Iqoih2BURUpIiKBZWigCihSm/P749zA8OQZIZk\nMjOZPG9f95Xcc245M5hnzpx7iswM55xz6ZOX6QI451xV44HXOefSzAOvc86lmQde55xLMw+8zjmX\nZh54nXMuzTzwOudcmnngdVlPUi1JwyQtkvRCOa7TU9LbqSxbJkh6Q9LpmS6HKzsPvGkmabqkVZIK\n4tI/l2SSWkX7g6P9PWOOaS3JYvZHSFohaUkUlEZJ2iXJcuwo6TlJcyUtlvS9pPskbRtzzFWSpkXX\nnynp+QTXPFnSuOj4X6IAsV9y70ypTgS2AhqZ2V/KehEz+6+ZHZ6C8mxE0oHRv9WQuPSOUfqIJK8z\nQNLTiY4zs25m9p8yFtdlAQ+8mTEN+GvRThQsaxVz3HzghgTXOt/M6gKNgBHAU4luLqk18CkwG9jV\nzOoDXYAfgf2iY04HTgUOja7fGXivlGteAtwN3EQIki2AB4DjEpUnCS2BKWa2JgXXqihzgX0lNYpJ\nOx2YkqobKPC/2VxgZr6lcQOmA32Bz2LS7gCuBgxoFaUNBu4C5gAHRGmtwz/Z+vNGAGfG7O8ErEqi\nDE8DwxIcMxC4O8nX9AdgCfCXUo6pSQjMs6PtbqBmlHcgMBO4FPgN+AXoHeVdC6wCVkf3+BswAHg6\n5tqtovcuP9rvBUwFfid8yPWMSR8dc96+wGfAoujnvnHv7fXAR9F13gYKSnhtReUfBPw9SqsWpfUD\nRsQcew/wM7AYGA90jdKPjHudk2LKcWNUjuXR/wPr/92BB4EXY65/K+EDUpn+f923kjf/9MyMT4D6\nktpJqgacRAiG8ZYRapA3JrqgpBpAz+jaiRwKvJREGU+TdLmkzlE5S7IPsAXwcinHXA3sDXQCOgJ7\nEj6AimxNCODNCMH1fkkNzaw/4T143szqmtljpRVaUh3gXqCbmdUjBNfPizluS+D16NhGhA+51+Nq\nrCcDvYEmQA3gstLuDTwJnBb9fgQwmfAhE+szwnuwJfAM8IKkLczszbjX2THmnFOBPkA9YEbc9S4F\nOkjqJakr4b073aIo7LKTB97MeYrwR3oY8C0wq4TjHgJaSOpWQv69khYSaknnE2qIiRQQatIASDpf\n0sKobfYRADN7GriAEEBGAr9JurKE6zUCCq30poCewHVm9puZzY3KeWpM/uoof7WZDY9ezx+TeC3F\nWQfsLKmWmf1iZpOLOeZo4Hsze8rM1pjZs4R/h2NjjnnCzKaY2XLgf4SAWSIzGwNsKemPhH/bJ4s5\n5mkzmxfd807CN4FEr3OwmU2Ozlkdd71lwCmED46ngQvMbGaC67kM88CbOU8RalS9KOYPtIiZrSR8\n5b0eUDGHXGhmDQg1zmOAFyV1SHDvecA2MfcYGF3jbqB6TPp/zexQoAFwDnCdpCNKuF6BpPxS7tmU\njWtrM6K09deIC9zLgLoJXscmzGwp4RvEOcAvkl6X1DaJ8hSVqVnM/pyY35Mtz1OED8CDKOYbgKRL\nJX0TPQxdSKjlF8QfF+fn0jLNbCyhaUWEDwiX5TzwZoiZzSC0Px4FDElw+BOEP9DjS7neOjP7EPgB\nSPTk/j3gz5tR1tVm9gLwBbBzMYd8DKwAupdymdmEh2RFWrDp1/BkLQVqx+xvHZtpZm+Z2WGED5dv\ngUeSKE9RmUr65pGsp4DzgOFRbXS9qCngn8D/AQ2jD7tFbPhALal5oNRmA0l/J9ScZwNXlL3oLl08\n8GbW34CDo1paiaKa4ADCH22JJO1DeMBW3FfrWAOArpLuktQsOrcAaBdzrV6SjpZUT1Je1NTRntAb\nIr58iwgPke6X1F1SbUnVJXWTdFt02LNAX0mNo3v1o/h27WR8DuwvqYWkPwD/iin3VpL+FLX1riQ0\nWawt5hrDgTZRF7h8SScR3rvXylgmAMxsGnAAoU07Xj1gDaEHRL6kfkD9mPxfgVab03NBUhtCz5dT\nCE03V0gqtUnEZZ4H3gwysx/NbFyShz9LeNofb2DUNruEUNvqa2ZvJLjvFMKDrm2BSZJ+Jzw1nw1c\nEx22GLgK+AlYCNwGnGtmo0u45l3AJYQHZnMJX4/PB4ZGh9wAjCPUmr8EJpC4q1xJ5X8HeD661ng2\nDpZ5hAdOswnd8Q4g1EDjrzGP0DRzKaGp5ArgGDMrLEuZ4q492syKq82/BbxB6GI2g/AtIbYZoWhw\nyDxJExLdJ2raeRq41cwmmdn3hH+zpyTVLM9rcBVL/vDTOefSy2u8zjmXZh54c1Q0XHdJMdtVmS6b\nc1WdNzU451yaldbvMqcpv5apRr1MFyMrdWjbPNNFyFpeTynZzz/NYP68wuL6mpdZtfotzdYsT3ic\nLZ/7lpkdmcp7V6SqG3hr1KPmH/8v08XISu99eHemi5C1Vq5el+kiZK1uB+2T8mvamuVJ/Z2u+Pz+\nRINQskqVDbzOucpAkIMTsnngdc5lLwF5pc3PVDnl3keJcy63SIm3pC6j86OJ+ldKGhyXV1vSA5IK\nixYViMmTpFslzYu226QNN5XUSdJ4ScuinwlHDnrgdc5lsaipIdGWnNmE0ZKPF5P3MGGqznbRz4tj\n8voQ5iHpCHQgjHg8G9ZPx/oKYQRhQ+A/wCtReok88DrnsluKarxmNsTMhhKGiMdcXn8E/gT0MbO5\nZrbWzMbHHHI6cKeZzTSzWcCdhFkFIUyCn09YNGClmd1LaCA5uLSyeOB1zmUvkcoab0n2IsydcW3U\n1PClpBNi8tsDk2L2J0VpRXlfxE08/0VMfrH84ZpzLosp2YdrBZJiJ5x62MweTvIm2xKmO32JME/z\nPoTVSL42s28I8zAvijl+EVA3aueNzyvKL3WQgAde51x2S64podDMOpfxDssJK6DcEE3BOlLSB4R5\nrb8hTC0aO31nfWCJmVk0K2D9uOvVJ6zTVyJvanDOZbGUPlwryRcJ8icTHqwV6ciGOa8nE9a8i/10\n6ECCObE98DrnspdIZXeyfElbEFaAriZpi2hO41GEeaf/FR3ThfDQ7K3o1CeBSyQ1k9SUMIfz4Chv\nBGGi/Qsl1ZR0fpT+fmll8cDrnMtuqavx9iU0K1xJWLFjOWHhgNXAcYRluBYRloo6zcy+jc57CBhG\nmMD/K8Lq1A8BmNkqQlez0wgLBpwBdI/SS+RtvM65LCaolpqRa2Y2gLDsVXF5kwkP1YrLM8IKJcWu\nZ2dmE4HdN6csHnidc9mrqDtZjvHA65zLbkm24VYmHnidc1nMZydzzrn08xqvc86lkZIeuVapeOB1\nzmU3b2pwzrk086YG55xLJ3+45pxz6ec1XuecSyMJ8nIvTOXeK3LO5Rav8TrnXJp5G69zzqWZ13id\ncy6N5L0anHMu7ZSXe4E3915RlhjUvycz3ruZcS9ctVH6uT0OYNLL1zD+xau58aLjAGixzZbM//gu\nPnnuSj557kruvbrHJtd74e6zN7lWLrjw3DNp26op++3RaZO8gffcRUHd6swrLNwofcL4z2hSvyav\nvvxSuoqZEZec34cOO27Lwfvsuj5t8pdfcOzh+3PIvrtxeo/j+X3xYgBWrVrFxX8/i0P23Y1D9+vM\nmNEjM1XslAoLUCjhVtl44K0gTw37hOP+fv9Gaft33pFjDtyFPf7vZnY/8UbufvK99XlTZxayd49b\n2LvHLVx443MbnXfcwR1ZumxlWsqdbj16ns7zQ1/bJH3WzJ8Z+f67bNu8xUbpa9eu5bprruLgQw9P\nVxEz5v/+eir/fXHYRmmXX3QOV/W/gffGTKDbMcfx4H13AfDMfx4D4L0xE3ju5eFc1/efrFu3Lu1l\nTjkluVUyHngryEcTfmT+omUbpfX5S1fueOIdVq1eA8DcBUsSXqdOrRpceMrB3PLomxVSzkzbd7+u\nNGy45Sbpff95Gf1vuHmT2swjgwZy7HHHU9C4cbqKmDF7d+lKg4YNN0r78Ycp7L1vVwC6HngIw4e9\nDMCU775hv/0PAqCgcRPq/+EPTJo4Pr0FrhCJa7vJ1nglnS9pnKSVkgaXcEx/SSbp0Ji0mpIel7RY\n0hxJl8Sdc4ikbyUtk/SBpJaJyuKBN41at2xCl113YNSTl/H2oxex+04banOtmjXi42f/yduPXkSX\nXXdYn97/vGO456n3WLa81CWccsobrw9jm6ZN2XmXjhul/zJ7Fq+/+gq9zjw7QyXLvD+2bc/bb4Ra\n8GuvvMTsWTMB2GnnDrz1xjDWrFnDTzOm8eXnE9fnVXYpbGqYDdwAPF7CfXYATgR+icsaAOwItAQO\nAq6QdGR0TgEwBLgG2BIYBzyfqCBZEXglTZe0XNKSmG1gTP6B0afQFXHntYrSi86ZLunK9L+C5ORX\ny6Nh/drsf9odXPXvoTx92xkAzClcTJtu/djnr7fyzzuHMPimXtSrswUd2jRj++aNefWDRKtP545l\ny5bx79tv5sq+AzbJu/qKS+l//U1US9EaXJXRXQMfYvCjgzjywL1ZumQJ1avXAKDHKb3Ypmkzuh20\nD/3/dRmd99yb/PzceJ/y8vISbskwsyFmNhSYV8IhA4F/AvG1nNOA681sgZl9Q1gMs1eU92dgspm9\nYGYrCEG6o6S2pZUlm3o1HGtm75aQdzowP/p5WzH5DcxsjaR9gPckfW5mWffdfNavCxn63iQAxk2e\nwbp1RkHDuhQuWML8RaH5YeI3PzN1ZiE7tmzC7u1bsNtOLfj29WvJr5ZH4y3r8dYjF3HEWfdk8mVU\nqOlTf+Sn6dM5YJ+wduDsWTM5eL89eXvkGD6fOJ6zep0CwPx5hbz71pvk5+dz1LHHZbLIadW6TVue\nHTIcCM0O7739BgD5+flce9Md64/70+EHsN32O2akjCmVfBtugaRxMfsPm9nDSd9G+guwysyGx9ag\nJTUEmgKTYg6fRFhZGKB9bJ6ZLZX0Y5T+LSXIpsBbLEm1CdX/s4AnJXU2s3HFHWtmH0uaDOwMZF3g\nHTbiCw7csw0fjv+e1i2aUKN6PoULllDQsC7zFy1l3TqjVbNGtG7RmGkzC5nw9U888sJoIPR8GHLv\nOTkddAF22nkXvp0+e/3+rju15t1Rn9CooIAJk79fn37+2Wdw+JFHV6mgC1A49zcKGjdh3bp13HPH\nLZza+ywAli9bhplRu04dRn3wLvn5+bRp2y7DpS0/kXRTQqGZdS7TPaS6wE1AcU9s60Y/F8WkLQLq\nxeTPjTsnNr9YWR94gROAJcALQE9CtX+TwKvwr7Mv4ZNmYnEXktQH6ANA9brFHZIy/7m5F11335GC\nBnX54c3ruX7QcP4z9GMeGtCTcS9cxarVazmz31MA7Ldba64592jWrF3L2rXGBTc+x4LFyxLcITec\n1esUPvpwJPPnFbJLm1b88+p+nHL6GZkuVlY472+n8vFHo5g/r5Dd22/PZVdew9KlSxj86CAAjjqm\nOyf1PB2AwsLfOPmEY8jLy2PrbZpy76BimzErpTR0F7sWeMrMphWTV/QEvD6wIub332Py68edE5tf\nLIUl4zNL0nSgAFgTk3y5mT0i6V3gKzP7h6S/AvcCTc1staRWwDTCJ4wBc4AHzezeRPfMq93Eav7x\n/1L7QnLEzNF3Z7oIWWvl6hzoolVBuh20D5Mmjk9plMxvtL3VP+qGhMcteLrn+GRrvJJuALY1s17R\n/ufAtmyIP40JMeVWM7tV0mzgdDN7Jzr+OqCNmfWIKnOnm1mXKK8OoQa8m5lViqaG7vFtvJKaE54i\n/itKegV4GDgaGBpzaIGZxQZt51wuECgvNbFcUj4h5lUDqknaghBsDwGqxxz6GXAJ8Ea0/yTQN2pD\n3orQ7Nk7ynsZuF3SCcDrQD/gi9KCLmRJr4ZSnEoo4zBJc4CpwBaE5gbnXBWQwu5kfYHlwJXAKdHv\nfc1snpnNKdqAtcACMytqZugP/AjMAEYCtxc9vDezuYTm0BuBBcBewKZDT+NkU423OKcR2l8GxaTt\nCbwgqVFmiuScS5fNeLiWkJkNIHT3SnRcq7j9lcAZ0Vbc8e8CpXYfi5dNgXeYpLUx+58BrYD7o0+V\nIq9K+gH4K7DpWFPnXE6pjHMxJJIVgTf+EyaJ49vH7Obev4pzboMc/AvPisDrnHPFEkmPTKtMPPA6\n57KaNzU451wapfLhWjbxwOucy265F3c98Drnspi8qcE559LOH64551y65V6F1wOvcy67eVODc86l\nUWVdRTgRD7zOuazmgdc559IsVdNCZhMPvM65rOY1XuecSyfvx+ucc+klIAfjrgde51w2y81eDbk3\nJMQ5l1Py8pRwS4ak8yWNk7RS0uCY9L0lvSNpvqS5kl6QtE1MviTdKmletN2mmE8DSZ0kjZe0LPrZ\nKeFr2ry3wDnn0kihqSHRlqTZwA3A43HpDQmL6LYCWhKWZn8iJr8P0B3oCHQAjgHOBpBUg7AI79PR\ndf4DvBKll8gDr3Mua4nU1XjNbIiZDQXmxaW/YWYvmNliM1sGDAS6xBxyOnCnmc00s1nAnUCvKO9A\nQpPt3Wa20szujYp9cGll8cDrnMtqSdZ4C6JmhKKtTzluuT8wOWa/PTApZn9SlFaU94WZWUz+FzH5\nxfKHa865rJbkw7VCM+ucgnt1APoBx8Uk1wUWxewvAupG7bzxeUX59Uq7jwde51zWkki6KaH891Jr\n4A3gIjP7MCZrCVA/Zr8+sMTMTFJ8XlH+76Xdy5sanHNZTOsnyiltK/ddpJbAu8D1ZvZUXPZkwoO1\nIh3Z0BQxGeigjQvRgY2bKjbhgdc5l9VS1atBUr6kLYBqQDVJW0RpzYD3gfvNbFAxpz4JXCKpmaSm\nwKXA4ChvBLAWuFBSTUnnR+nvl1YWb2pwzmW1FA6g6Av0j9k/BbgWMGB7oL+k9flmVjf69aEo/8to\n/9EoDTNbJal7lHYL8A3Q3cxWlVYQD7zOuey1ef10S2VmA4ABJWRfW8p5BlwRbcXlTwR235yyeOB1\nzmWton68ucYDr3Muq+XiXA0eeJ1zWS0H427VDbwd2jbn3VF3Z7oYWeng20ZmughZa8zVpY4ErdKq\nVUSTQFWbj1dS7WQvEo1vds65lKqK8/EuIXSzSEa1FJTFOefiJD8JTmVSWuDtlrZSOOdcCapUU4OZ\nvZXOgjjn3CZS2I83myQ9ZFhSo2gG939LahSl7SGpecUVzzlXlYU23oqfqyHdkurVIKkj8B4wF9gB\nuI8wmfCfCLO2n1pB5XPOVXGVMbAmkmyN907gUTNrB6yMSX8D6JryUjnnXCRVK1Bkk2T78e5BtMZQ\nnFnA1qkrjnPOxcjRNt5kA+9KNp3sF6ANUJi64jjn3Aaq4su7DwP6SioK1BbNYXkL8HKFlMw550jp\nKsNZI9nAexmwLTAHqEWY5HcqsAq4qmKK5pxzkCcl3CqbpJoazGyBpH2AI4HdCAF7AjDczNZVYPmc\nc1VYOtdcS6ekJ8mJAuzwaHPOubTIwbi7WQMojpT0tqSZ0faOJB9W7JyrUKkaQBENABsnaaWkwXF5\nh0j6VtIySR9Ei18W5dWU9LikxZLmSLok2XNLklTglXQB8CrwK3BTtM0Bhkq6MJlrOOdcWaTw4dps\n4Abg8Y2vrwJgCHANsCUwDng+5pABwI5AS+Ag4ApJRyZ5brGSbWq4ErjYzO6PSXtA0seEBeTuTfI6\nzjmXNBG6lKWCmQ0BkNSZ0FmgyJ+ByWb2QpQ/ACiU1NbMvgVOA3qb2QJggaRHgF7Am0mcW6xkmxr+\nABQ3ac7bFN+/1znnyk+iWl7iDSiImhGKtj6bcZf2wKSiHTNbCvwItJfUEGgamx/93j7RuaXdMNka\n7zDgOMLQ4VjH4Q/bnHMVKMmmhEIz61zGW9QlzEMTaxFQL8or2o/PS3RuiUpbgeK8mN0vCQMougKf\nRGl7AwcAt5V2A+ecKytBOvrpLmHTb+71gd+jvKL9FXF5ic4tUWk13mvi9lcAe0VbbNoFwM2l3cQ5\n58oqDeMjJgOnb7if6hBmYZwcjWH4BegIvBMd0jE6p9RzS7thaROhb1OGF+CccymVqrkaoikP8glL\nlVWTtAWwhjDtwe2STgBeB/oBX8Q8HHuS8I1/HLAVcBbQO8pLdG6xku7H65xz6SaR7MO1ZPQFlhN6\naZ0S/d7XzOYCJwA3AgsI3+p7xJzXn/DAbAYwErjdzN4ESOLcYiU9ck1SK0LXiRZAjdg8MzuvmFOc\nc67cUtXSYGYDCH1yi8t7F2hbQt5K4Ixo26xzS5LsChSHEQZQfMuG7hPbE6rsYzfnhs45tzmq8rSQ\nNwG3mNmuhLl5TyLUfEcSArJzzqVc6NWQeKtskg28bYGno9/XALWijsL9gMsromDOOUcS8zRUxhpx\nsoF3KRvadX8hdJcAMKBRqgvlnHNFqvKaa2OBfYFvCOOTb5fUjvA0z9t4nXMVoqipIdckG3gvZ8PQ\nuf5AQ+BvwBTCAApXigvPPZN33hxOQeMmfDj2cwBuu+k6nhr8GI0KCgC4uv8NHHZEN158/hkG3rNh\nZPbXX33Je6PHskuHThkpe0W49rh27N+mgPlLV3HCA58CcNhOTTj3oO3YrqAOPR/5jK9nh4E/R+2y\nFad32TDLXput6tLjobF8N2cJj/bajcb1arBidZiL/9ynJjJ/6er0v6A0GXjPvxn8xGNIov3OuzDo\nkcc57+wzmTh+HPnVq9N5jz249/6HqF69eqaLmlKVsSkhkaSaGszsOzMbH/3+u5n1NrM2wPGkrrdH\nzurR83See/m1TdLP+ftFjBgznhFjxnPYEWFq4xNPOnl92gOPDKZFy1Y5FXQBXvn8F859+vON0n74\nbQkXP/cl42cs3Ch9+Je/ctKgsZw0aCxXD5nM7IUr+G7OkvX5/3pp8vr8XA66s2fN4sH77+PDjz/j\ns4lfsnbtWl7833Oc1ONkJnz5DWMnfMHy5SsY/PijmS5qyimJrbJJuh9vCXYiLAFULQVlyVn77teV\nn2ZM3+zzhrzwPMefeFLqC5RhE2YspGmDLTZKm1a4LOF53XbZmje+mlNRxcp6a9auYfny5VSvXp3l\ny5axzTZNOeSww9fnd+68B7NmzcxgCVNPSstcDWnnI9cy6LGHH+CAvXflwnPPZOGCBZvkvzLkBf78\nl9wLvGV1xM5NePPLXzdKu677Tjx/zp70OaBVZgqVJk2bNePCf1xKu9Yt2aFlU+r/4Q8bBd3Vq1fz\n7DNPc9jhR2awlBUjFx+uZUXglTRd0qFxab0kjY7JXy5piaRfJT0hqW6UN0LSmZkod3n0OvNsPvvi\nOz4YM56ttt6Gfldt3Ctv/GefUqtWLdrttHOGSphddmlWnxWr1/HDb0vXp1310mROfOBTej8+nt1a\nNOCYjltnsIQVa8GCBbz+2qt89d1Ufpg+i2VLl/LcM0+vz7/4wvPosl9XuuzXNYOlrBhVeXn3bHCs\nmdUlrHK8B2HcdaXVpMlWVKtWjby8PE7t9Tcmjh+3Uf7LL/2P409MOOS7yjhil61448uNmxl++30l\nAMtWrWX4l7+yS7PcnZP/g/ffpVWrVjRu3Jjq1avzp+7H88nHYwC46YZrKZxbyC2335XhUqaeSLy0\ne2Vsiii1jVfSTgnO3yFBfsqZ2SxJbwCVuio4Z84vbL11mABu+LChtN1pw4T169at49WXX+LVN9/P\nVPGyigSH79SE3k+MX59WLU/U2yKfhctWk58n9m9TwKdT52ewlBWrefMWjP30U5YtW0atWrUY8cH7\n7Lrb7gx+/FHee+dtXnvzXfLyKlM9KkmVtEabSKKHa18RBkmURAnyU05Sc+AowgJzm3tuH6APwLbN\nW6S4ZCXr0/sUPvpwJPPnFdLhj6244qp+jBk9kq++mIQkmrdoxR33PrD++I8/+pCmTZvRarvt01bG\ndLrlxPZ0btWQBrWr8/YlXXhwxFQWLVvDlUe1oWGdGgzs2Ynv5vzOuU+Fng+7t2zAr4tXMmvBivXX\nqFFNPHhqJ/Kj2ak+mbqAl8bPytRLqnB77LkX3f98Al322p38/Hw6dtqVM87sQ5OGdWnRoiUH778v\nAH/qfjz/urpfhkubWrnYnUxmJcdNSX9M5iJm9l25CiFNBwoIw5GL1AAmmNl+cfmLCPNeXmpmyyWN\nAJ42s83qR9Npt93t3VGflqfYOeuQ20dmughZa8zVB2e6CFmr6z57MGH8uJRGya1a72wn3fFiwuPu\nO77d+HIs/ZN2pdZ4yxtQN1P3aHo1IDxcA84sKd85VzVUwk4LCZW3H69zzlUoD7zZKz9axqPIWjPL\n3WFMzlURobtY7kXeXHkM+iBhGY+i7YnMFsc5lyqpmo9XUitJwyUtkDRH0sBoHTYkdZI0XtKy6Gen\nmPMk6VZJ86LtNpXz0yArarxm1qqYtMHA4JLyY447sGJK5ZzLNMHmrKmWyAPAb8A2QAPCqsHnSRoE\nvALcHR1zNvCKpB3NbBWhJ1R3wurCFp03FRhU1oJsVo1XUl1JHSXl1vRHzrmslZfElqTtgP+Z2Qoz\nm0OY4rY9cCChEnq3ma00s3sJMb+oC8vpwJ1mNtPMZgF3Ar3K+5oSklRH0pPAYmA80DxKHyjp6vIU\nwDnnSpPkkOECSeNitj7FXOoeoIek2pKaAd3YEHy/sI371n4RpcOGdSaLTIrJK5NkPyxuJiz/sy+w\nIib9beAv5SmAc86VREkMF46GDBeaWeeY7eFiLjeSEDAXAzOBccBQwlzji+KOXQTUi36Pz18E1C1P\nO2+ygfc44AIz+4SNR6p9TVht2DnnKkQqJsmRlAe8RRjxWocwIKshcCuwBIif6KM+8Hv0e3x+fWCJ\nlTb6LIFkA29jQqN0vDplvbFzziUiID9PCbckbEloIh0YtePOI/R+OgqYDHSIq8F2iNKJfnaMyesY\nk1cmyQbe8VEBixRF+jOAj8tTAOecK00qarxmVghMA86VlC+pAeGh2SRgBLAWuFBSTUnnR6cVzVL1\nJHCJpGaSmgKXEvW4Kqtku5MkKgAoAAAb1klEQVRdDQyX1DY65++Sip4GHlCeAjjnXIk2o59uEv5M\n6DL2T0Kg/QC42MxWSeoOPArcQljUt3vUlQzgIUKT6pfR/qNRWpklFXjNbJSkA4ArgFnRC5gAdDGz\nCeUpgHPOlUYpWlXNzD4nVBaLy5sI7F5CnhFi3xUpKQibMYAiWuzS16FxzqVNlV7eXVLt0vLNLPFK\nhc45VwYpHLmWNZKt8S6h9AnPfZVh51zKVekaL2GER6zqwK6E+XKvSWmJnHOuSBVd+gcAM3urmOTX\nJE0BTiF0t3DOuZSrjItZJlLeaSHHsWEiCeecS6mipoZUTAuZTco8LaSkGsDfCd3LnHOuAohqOVjj\nTbZXw1w2frgmwnyWq4DTKqBczjmHqMJtvEDfuP11wFxgjJkVN4eDc86VXyVtSkgkYeCNlsZYDQyP\nJg92zrm0qZIP18xsDTAQqFnxxXHOuQ2KmhrKO0lOtkm2qWEsYSq0GRVYFuec20RVHrk2ELgzmhJt\nPLA0NtPMvk51wZxzTuTOUuixkg28/4t+PhD9LOrhoOh3HzLsnEs9heV/ck2ygbddhZbCOedKkHth\nN0HglfQ4cJGZfZem8jjn3Hph5Fruhd5EzSenA7XSURDnnCtOKocMS+oh6RtJSyX9KKlrlH6IpG8l\nLZP0gaSWMefUlPS4pMWS5ki6pNyvKVE5y3sD55wrOyEl3pK6knQYYVXh3oSl2/cHpkoqIKw+fA1h\nUcxxwPMxpw4AdgRaAgcBV0g6sjyvKpk23jIvYeycc+WR4l4N1wLXmdkn0f4sAEl9gMlm9kK0PwAo\nlNTWzL4lTIvQ28wWAAskPQL0At4sa0GSeU1zJK0tbSvrzZ1zLpFU1HglVQM6A40l/SBppqSBkmoB\n7QmrDQNgZkuBH4H2khoCTWPzo9/bl+c1JVPj7QMsLM9NnHOurJJs7yyQNC5m/2EzezhmfyvCAg4n\nAl0J0yC8QpiHpi5h7plYiwjNEXVj9uPzyiyZwDssVyfCCYuHunifXnNIpouQtdpd/nqmi5C1Zs9c\nlPigzSSR7LSQhWbWuZT85dHP+8zsl3Bt3UUIvKOA+nHH1wd+Jyx7VrS/Ii6vzBI1NXhkcs5lVCqa\nGqL22ZkUH9MmE6ZEKLpfHWAHQrvvAuCX2Pzo98nleEneq8E5l92UxJakJ4ALJDWJ2m7/AbwGvAzs\nLOkESVsA/YAvogdrEJY26yupoaS2wFnA4PK8plKbGswsF4dJO+cqkRSOn7geKACmEJoN/gfcaGYr\nJJ1AmJPmaeBToEfMef2BBwmThC0HbjWzMvdogHIs/eOccxUtdCdLTeQ1s9XAedEWn/cu0LaE81YC\nZ0RbSnjgdc5lMeXkkGEPvM65rJaDcdcDr3Mue6WyqSGbeOB1zmWvSrq0TyIeeJ1zWc0Dr3POpZFI\neuRapeKB1zmX1eRtvM45l145WOH1wOucy25e43XOuTQKa65luhSp54HXOZe95CPXnHMu7XIv7Hrg\ndc5lsVxd3t0Dr3Muq+Ve2PXA65zLdjkYeT3wOueymjc1OOdcmuVe2E285ppzzmVWChddA5C0o6QV\nkp6OSTtZ0gxJSyUNlbRlTN6Wkl6O8mZIOrm8L8kDr3Mua4W4mvi/zXQ/8Nn6e0jtgYeAU4GtgGXA\nA3HHr4ryegIPRueUmTc1OOeyV4rn45XUA1gIjAFaR8k9gWFmNio65hrgG0n1gHXACcDOZrYEGC3p\nVUKQvrKs5fAar3Muq0mJN6BA0riYrc+m11F94Drg0ris9sCkoh0z+5FQw20TbWvNbErM8ZOic8rM\na7zOuSyWdFNCoZl1TnDM9cBjZvazNq5G1wUWxR27CKgHrC0lr8w88DrnsloqmhokdQIOBXYtJnsJ\nUD8urT7wO6GpoaS8MvPAmwYXnXcW77w5nILGjRn16efr0x8ddD+PPfwA+fn5HHpEN/pffwsTxn3G\npRedC4CZcfm/ruHoY7tnquhpt3DhQs49+0y+nvwVkhj08OO8MnQIw18fRo3qNdhuhx14+NEnaNCg\nQaaLWiFu7dGBg3dqwrwlqzjytlEA/OvYthzSfitWr13HjMJlXP7sJH5fsYZmDWvx7pUHMHXuEgAm\nzlhI3xe+AuDoTttw/mGtycsTH3z9G7cM+zZjr6k8ytBpoSQHAq2An6Labl2gmqSdgDeBjuvvKW0P\n1ASmEAJvvqQdzez76JCOwOTyFMbbeNOgR8/TeG7IaxuljR41gjeGD2PExxP4cOwkzrvwEgDa7tSe\nd0Z+wgcfjeP5Ia9x+UV/Z82aNZkodkZcdvFFHH74kUz66lvGjp9E23btOOTQwxj/+Vd8NvELdtyx\nDbffenOmi1lhXho7k14Pj90obfSUQo64bRTdbv+QaXOXct6hrdfnzZi3jKPvGM3Rd4xeH3Qb1K7O\nv/7Ujp4PfMoRt46ioF5N9t2xUVpfR0qlpjvZw8AOQKdoGwS8DhwB/Bc4VlJXSXUI7cBDzOx3M1sK\nDAGuk1RHUhfgOOCp8rwkD7xpsE+XrjRo2HCjtMGPPcSFF19OzZo1AWjcuAkAtWvXJj8/fBFZsWJF\nbk6/X4LFixczevQoep3xNwBq1KhBgwYNOPSww9e/J3vutTezZs7MZDEr1Nip81m4dPVGaR9+V8ja\ndQbAxBkL2LrBFqVeo0Wj2kybu5T5S1cB8NGUQo7ssE3FFDgN8qKpIUvbEjGzZWY2p2gjNC+sMLO5\nZjYZOIcQgH8jtN+eF3P6eUCtKO9Z4NzonLK/pvKc7Mruxx++55MxoznyoC4c1+0QJo4ftz5v/Gdj\n6bpnRw7YZzduv3vg+qCT66ZNnUpBQWP6/K03e3felXP7nMnSpUs3OubJwY9zxJHdMlTCzPu/vZoz\n8pu56/ebb1mL1y7dj+f+vjd7bB8+3KcXLmWHJnVo1rAW1fLEYTtvRdOGpQfrbJbi8RMAmNkAMzsl\nZv8ZM2thZnXM7Dgzmx+TN9/Mukd5LczsmXK9INIYeCVNl7RKUkFc+ueSTNI+khZLqhaT90gJaYOi\n39tLelvSAkkLJY2XdFS6XlN5rF2zhkULF/LG+6Ppf/0tnNXrZMxCrWb3Pfbkw7GTeHvEGO6987ZQ\n860C1qxZw+cTJ3DW2efyybiJ1K5Thztuu2V9/q0330i1/Hx6nNwzg6XMnL8f2po1a42h42cBMHfx\nSrpc9z7H3DmaG175mrtP2ZW6NfNZvHwN17z4FQNP35X/XbAPs+YvZ81ay3DpyyiZqFsJvxSmu8Y7\nDfhr0Y6kXQhVeIBCoBqwW8zxXYHZcWn7A6Oi34cB7xBGlDQBLgQWV0TBU22bptty9J+6I4ndOu+B\nlMe8eYUbHdPmj+2oXacO335drm81lUazbbel2bbbsudeewFw/Akn8vnECQA8/eR/GP76awx+8r+o\nCjW/FPnzHs04uH0T/vH0xPVpq9auY+Gy0Czx1czF/DRvGds1qQPAe5N/4/i7x3DCPWOYOncJ0wuX\nFnvdyqACRq5lXLoD71PAaTH7pwNPRr+vBj4hBFYkNQFqAM/HpbUBRkU15+2AR8xsVbR9ZGaj0/JK\nyqnbMX/iw5EfAPDj91NYvXoVjRoVMGP6tPUP037+aQY/fD+F5i1bZrKoabP11luz7bbNmfLddwCM\neP892rbbibffepM777iVF19+ldq1a2e4lOm3f9vGnHPwDpz16DhWrF63Pn3LOjXWr0fWvFEtWhXU\n4ad5ywBoVLcGAPVr5XNKl5Y8/8nPaS93KoikB1BUKuluPPwEOFVSO0JXjZOA/YAbovxRhCB7Z/Rz\ndLSdH5M2zcxmKlR7fgCelvQo8LGZ/VrazaPRLH0Atm3eIsUvrWRn9z6Fj0aPYv68Qjq23Y4rrurH\nyaf24qLzzmL/vTpRvUYN7hv0GJL49OOPuO/ft5NfvTp5eXncete9NGpUkPgmOeKuu++j92k9WbVq\nFa22356HH32C/fbZg5UrV3LMkYcB4QHbfQ8MynBJK8Y9p3Zi79aNaFinBmP6H8zdb37PuYfsQI38\nPJ46d09gQ7exPXfYkou7tWHtWmOtGX1f/JJFUQ243/E70a5p6H5679vfM21uJa7xVsLAmoiK2hUr\n/EbSdOBMYG+gDjCSMHSvG6G2ux2hn91LQAFwN/AV4Snij8DWUVp9M+sdXXNbwnjpbtH5o4G/xfS3\nK1Gn3Xa3d0Z+krLXl0vq1aqe6SJkrXaXv57pImSt2c/8g5W/fp/SMLlzx93sxTcTf4lt17TO+CRG\nrmWNTPRqeAo4GejFhmaGIp8QOjbvTKjdfhhNTPFzTFpR+y5mNtPMzjezHYCWwNJirumcq8Rysakh\n7YHXzGYQHrIdReiYHJu3gjBd2zHANmZWNNzmwyitAzGBN+7cnwnTt+1cMSV3zmVCDnZqyFg/3r8B\nB0ejQuKNAv5BmLatyOgobU40cxCSGkq6VlJrSXnRw7YzCLVm51yuyMHIm5HAa2Y/mtm4ErJHErqG\nxTbsjI7SYmu7qwhtwu8SupB9BawkNGE453KAlJqRa9kmbb0azKxVCelriPnMMrO3iPsMi3orxKct\nJXRHc87lsMoXVhOrGmNRnXOVVw5GXg+8zrksVjlHpiXigdc5l9UqYRNuQh54nXNZq2jIcK7xwOuc\ny2re1OCcc2nmNV7nnEuzHIy7vgKFcy6LCSQl3BJeRqop6TFJMyT9LmmipG4x+YdI+lbSMkkfSGoZ\nd+7j0aIMcyRdUt6X5YHXOZe1Ujgfbz5hsq0DgD8A1wD/k9Qqmm5gSJS2JTCOMA94kQHAjoSJuA4C\nrpB0ZHlelzc1OOeyWiqaGqKRrgNikl6TNA3YHWgETDazFwAkDQAKJbWNJuo6DehtZguABZIeIUxN\n8GZZy+M1XudcVkuyxlsgaVzM1qf0a2orwmo2k4H2wKSivChI/wi0l9QQaBqbH/3evjyvyWu8zrms\nlmR3ssJkJ0KXVJ2wlPt/zOxbSXWBuXGHLSIs8143Zj8+r8w88Drnsloqu5NJyiMsxrCKsKQYwBKg\nftyh9YHfo7yi/RVxeWXmTQ3OuayVTDNDsoE5WqfxMcKq5CeY2eooazLQMea4OsAOhHbfBcAvsfnR\n7+Va+tsDr3Muq6VwefcHgXbAsWa2PCb9ZWBnSSdI2gLoB3wRswLOk0DfaPGFtsBZwODyvCYPvM65\n7JaCFSiifrlnA52AOZKWRFtPM5sLnADcCCwA9gJ6xJzen/CwbQZhoYbbzazMPRrA23idc1kuRd3J\nZpR2KTN7F2hbQt5KwrJiZ6SgKIAHXudcVqucS/sk4oHXOZe1cnVaSG/jdc65NPMar3Muq+VijdcD\nr3Muq/lE6M45l0YS5OVe3PXA65zLch54nXMuvbypwTnn0swfrjnnXJrlYNz1wOucy27JrKlW2Xjg\ndc5lrVwduSYzy3QZMkLSXMJsQ9mgACjMdCGylL83pcum96elmTVO5QUlvUl4jYkUmlm5FqBMpyob\neLOJpHHJLltS1fh7Uzp/fyonn6vBOefSzAOvc86lmQfe7PBwpguQxfy9KZ2/P5WQt/E651yaeY3X\nOefSzAOvc86lmQde55xLMw+8WUi5OEbSObeeB94sZP7E05WRpGMlFbtMucsePldDFpF0FrA18Ckw\nzszmZ7hIrhKR9DzQGLhR0jQzW5npMrnieXeyLCFpKNAMWAI0Ae4xs4clyWvALpEo6DYFjgaWm9nq\nuPw8M1uXkcK5TXhTQxaQNAyoZWZ7mNlBwEjgbEn5VT3oSjpRUvtMlyObSeoMtACONLPFZrZaUmNJ\nB0g6DMDM1knyv/cs4U0NGSbpIEIt5cCY5FcItd86wKIMFCsrSNoZuB54VdJjZjYl02XKUgbUM7Ol\nkpoAuwFPAguAhpJGmdmJXuPNHv4JmGFm9gHQH3hLUidJ1YFHgPfNrMoGXQAz+wo4Ddgb+JukNkV5\nkqrF/P6HDBQv4yRdLOkMMxsPbCHpC+AJ4N/AfcABwMnAnkU1X5cdvMabQUXtbmZ2ffQ1cBywHLjK\nzO6LjqlybbySbgJ2AOYCdwMXEQKJSXrCzL4zs7XRsRcBHSWdY2arMlboNJNUB9iLUHl6nFDLvQGY\nBEwzs/ej45YR5p3+PUNFdcXwh2sZIGl3YIqZ/R6XfgFwD7CzmX0tqXr8Q5JcJ+kVwsTX7wGdCIGl\nO6H21h/4GHjczL6TdBlwG7C7mU3MUJEzRtIOwHjgdjO7sYRjzgb+ARxqZrPSWT5XMq/xppmkG4Cr\ngNGSbgS+MLNfAMzsPkmNgc8lHWFmH1Slp9GSXgSaFU3sLak7cAuwjZm9J8mAAcDCqC3zHGCPqhR0\nJZ0DPArkmdmPks4FBkj6uKiWGx23H9AFuBw43INudvHAm35fAdOA+cBgYKKk94GBwEoz6ydpLfCe\npP3NbHTmipo+krYEjgDukVTbzJYBXwPLgLpRk8v70aC+ewkPH/eL2jerBEl/Bh4gtHm/K2kU8ALh\nfesmabyZLYqaIbpFxx1kZl9mrNCuWN7UkCaStjCzFZK2Bz4ALgMmAKcAvQjtmW8QvjYukXQ+8J6Z\nfZOpMqeLpH6EJoSZwKuED6Q3gKHAQDO7Le74zoQ1tqant6SZJak54T1ZSgi43Qg12m0JH9y9zOyj\n6NhahC6KPggnC3mvhjSQ9AjQE8DMpgJ3ENom15nZtYQgvDtwIvCLpEuAB6tI0G0I1AJuBLYEjgPO\nAMYA/y0KurF9UM1sXFULugBm9jNwAVANmAr8DxgG1ABmA/dLqhcdu9yDbvbywFvBoodFu5rZYzHJ\nHwC/Ai0k7U/o/nMesAuh/Xdo0VP7XGdmC4D7CX2X7wIaAocDswhtuVtGx1WJdu54kq6R9LGk1lET\nzBjgNWAnMxsMnA8cSlhpuAPh/yOX5bypoQJJehlobGb7xaTVMLNVkh4ATiUMEb7azB7PVDkzIRqV\ntyZmvzpwE3AM0BtYTKjNPUsYPj03IwXNIEn/JIxivBGoDrwLPEiYz+MW4GIzmyKpBbAjoTtZbzP7\nNkNFdknywFtBJD1KeOjRKqbP6faEB0PnAyuBd4BRZnZeVeqvK+lmwkPGF4FVZmaSmhLavL8C6gGX\nEmpxHxNqwjdWpVpv1MZ/L/A20Ac4FjgEaAf0IAws2Rc4oKj/clXqAVPZeVNDxXmK8ET+cABJ2wKj\ngM+j9smFwERC7aWqTQX5AdAPODYKus0JM7LdQ/iwGkpoA9+GMEjg2SoYUF4HPgT2JDQtvE7oPvc6\nIRjPJzTLnFM0f3MVfI8qLQ+8FcTMRhJqKndFfS8/Bh4ws75R7XY5IdgcJKlBVZr83MzeBs4Cro/e\nmzHAIDO7Ofp28AzwPtAX+NnMfshcadMvGg49mxBkTyd8QI8C6pjZZYRvTO2ANsCfgS0yVFRXRt7U\nUMGih2fDgdfMrEdM+sWEr4s9zOy7TJUvk4p7b4q+LkdNDyvNbF5GC5lGkvYBxsY0TfUGLibUeh8i\nPETbz8ymSWpG6Kf7VVX9/6cy88CbBpK6EEYbXW1mQ6Ja3s2EYZxVZgBAcaL35hHgauCtaOBElSNp\nMOGD+FFCj5f+0QfQLcAvZnaPwpzNHQgj0arUt4Bc44E3TSQdQJjo5XPCNJCHmdmEzJYqO0TvzQOE\np/IvVaXJbopI2pfQdvsksB0g4C1CO3cNM/tHdNw7hOcCu8b2CnGViwfeNIrm3n2GMGH1pEyXJ5tI\nOpTwLeDg+MmDqoqo9j8IuIQwsXk9Qjv3lsARZvZOdNy2ZjYzYwV15eaBN82Khg5nuhzZKGaOhior\n+nC+F+hnZi9L6gS0JgyhXu49F3KDB17nsoykAwnt3v8ysxczXBxXAXx2MueyjJmNkHQm8GA0oq9K\ntnvnMq/xOpelvN07d3ngdS6Lebt3bvLA65xzaeZDhp1zLs088DrnXJp54HXOuTTzwOucc2nmgdc5\n59LMA68rM0lfSRoQsz9d0mUZKEdnSSapVQXeY4Ckr1JwnYy8Ry67eODNIZIGRwHIJK2WNFXSHZLq\npKkIexBmGUtIUi9JSyq4PLH3GyFpYLru51xpfMhw7nmXsIhmdaArYX7XOsC5xR0sqbqZrU7Fjavi\ngpTOlYXXeHPPSjObY2Y/m9kzwH+B7hAmX4lqw0dJGitpFWGNMyQdK2m8pBWSpkm6UVKNootKaiLp\nFUnLJc2QdEb8jeO/RkuqL+lBSb9E1/1G0knRJDBPAHViaugDonNqSLpV0kxJSyV9JumIuPscKenb\n6JofEpbAKRdJt0j6Lnp90yXdJmmTJXUknSnpp+i4oZIK4vJ7S/o6KtsUSRdL8r8ztxGv8ea+5YTa\nb6xbCav4/gD8HgW2/wIXEdb2akGYF7YmUBRIBwMtCcvPLAP+DbQq6abRGnJvEBZk7A1MAf5IWB9s\nDPAPwnLuO0SnFDU7PBGlnQzMBI4Chknaw8wmRQtjDiXM3nU/YUWGu5J/O0q0FDgDmAXsRHj9K4Fr\nYo5pBZwCHAfUBh4GHgf+FL3ms4DrgAuA8cDOUTlXA97M4TYwM99yZCMEx9di9vckLJH+fLR/IGDA\nCXHnjQKuiUvrTgiGItQoDegSk98SWAsMiEmbDlwW/X4YsA5oV0JZewFL4tJ2iM5pEZc+lLBQKIRg\nPYVouHuU1jcqX6tS3psRwMDNeC/PAX6I2R8Qvd4WMWn7RffdMdr/CTg17jr/AL4u7j3yrepuXuPN\nPUdGD63yCTXdVwg1sFjj4vZ3B/aU9M+YtDygFmGZmXaEgDi2KNPMZkiaXUo5diWsFfbNZpR9N0Kg\n/zpu0eWahFWHicryiZnFTjLy8Wbco1iSTiQEydZAXaBatMWaZWY/xex/SvThImkh0Bx4SNKDMcfk\nE16Tc+t54M09owjLyq8GZlvxD86Wxu3nAdcCLxRz7FzKFjjKck4eoQa5B6H8sZaX47qlkrQ38Bzh\nPbgYWEhoPrhjMy5T1I5btFy9cyXywJt7ltnmr0A7AWhb0nmSviEElj2IgoqkFkDTBNfcRlK7Emq9\nq9i0RjmREFi3NrMPSrju18AJkhRT6927lHIkowuhNnt9UYKklsUc10xSczP7Odrfk/C+fGNmv0qa\nBexgZk+Wszwux3ngdRAeCL0maQbwP2AN4cHQnmZ2hZl9J+lNwtfoPoTa511sqIUW5z3CV/GXJF1M\naJdtDdQxs6GEts4tJB1GCLjLzGyKpP8CgyVdSgjeWxLapqea2RDCQ69LgbslPQDsQqhlJqMgWsMs\n1m9R2ZpJ6klotjgC+Gsx5y8H/iPpEkIzzCDgdTP7PsofANwXNTsMJzT17AY0M7ObkyyjqwK8m4vD\nzN4iLDl/EKEddyxwJeFhUZFewDRCW+swwmrJ00u55jqgG/AR8DTwDXAPUCPKH0MIXM8SmjOuiE7t\nTejZcBvwLfAasD8wIzrvJ+DPwJHAJELTwJVJvtSTCEE+drvEzIYBtwN3A18QHgz2K+b86YQmiWHR\n+zA1Km/Ra36U0DPi1KhsHxKafaYlWT5XRfhE6M45l2Ze43XOuTTzwOucc2nmgdc559LMA69zzqWZ\nB17nnEszD7zOOZdmHnidcy7NPPA651ya/T+pw3wPNXGeTAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2a395962c18>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Confusion Matrix\n",
    "conf_mx = confusion_matrix(y_test, y_pred)\n",
    "plot_conf_matrix(conf_mx, train.author.unique(), title=\"MNB_GS Confusion Matrix\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### SGD"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 333,
   "metadata": {},
   "outputs": [],
   "source": [
    "#SGD\n",
    "#clf__alpha: 1e-05\n",
    "#clf__n_iter: 50\n",
    "#clf__penalty: 'l2'\n",
    "#tfidf__norm: 'l2'\n",
    "#tfidf__use_idf: True\n",
    "#vect__max_df: 1.0\n",
    "#vect__max_features: None\n",
    "#vect__ngram_range: (1, 2)\n",
    "\n",
    "sgd_clf = Pipeline([\n",
    "    ('vect', CountVectorizer(stop_words='english', max_df=1.0, max_features=None, ngram_range=(1, 2))),\n",
    "    ('tfidf', TfidfTransformer(norm='l2', use_idf=True)),\n",
    "    ('clf', SGDClassifier(alpha=1e-05, n_iter=50, penalty='l2', loss='log'))\n",
    "])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 334,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\anoop\\Anaconda3\\lib\\site-packages\\sklearn\\linear_model\\stochastic_gradient.py:117: DeprecationWarning: n_iter parameter is deprecated in 0.19 and will be removed in 0.21. Use max_iter and tol instead.\n",
      "  DeprecationWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Pipeline(memory=None,\n",
       "     steps=[('vect', CountVectorizer(analyzer='word', binary=False, decode_error='strict',\n",
       "        dtype=<class 'numpy.int64'>, encoding='utf-8', input='content',\n",
       "        lowercase=True, max_df=1.0, max_features=None, min_df=1,\n",
       "        ngram_range=(1, 2), preprocessor=None, stop_words='english',\n",
       "        ...='l2', power_t=0.5, random_state=None,\n",
       "       shuffle=True, tol=None, verbose=0, warm_start=False))])"
      ]
     },
     "execution_count": 334,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sgd_clf.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 335,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = sgd_clf.predict(X_test)\n",
    "y_pred_prob = sgd_clf.predict_proba(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 336,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy score: 0.794484167518\n",
      "Log loss: 0.524887624206\n"
     ]
    }
   ],
   "source": [
    "#create_submission_file(y_pred_prob, 'SGD_GS')\n",
    "print(\"Accuracy score:\", accuracy_score(y_test, y_pred))\n",
    "print(\"Log loss:\", log_loss(y_test, y_pred_prob))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 337,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV4AAAEqCAYAAABdmDjNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XeYVEXWx/HvbxiShAVEBRFBMYCw\ngAImBMUMK4phV9aIumZ916yriBgw55xWMe0aVkRBxKyIiggCKooIEiQpOSfhvH/UHehpZqaboae7\np+d8fO4z01U31G2c09V1K8jMcM45lz55mS6Ac85VNB54nXMuzTzwOudcmnngdc65NPPA65xzaeaB\n1znn0swDr3POpZkHXpfVJHWU9LOkZZJ6bMF53pF0eirLlm6Sdozeh0qZLovbMh5400zSAZK+kLRY\n0gJJn0vqEJPfUNJTkmZFf2S/SOovqXmU31SSRXnLJP0mabCkwzajDD0lfSVpuaTfo98vkKQofwdJ\nr0uaF5XzO0m9SjhfbUn3S5oelWlS9Lr+FrxVBW4CHjazmmY2sLQnMbOuZvZcCspTSPRvY5KOjku/\nP0rvleR5pko6tKR9zGx69D6s24IiuyzggTeNJNUGBgMPAfWARsCNwOoof2vgC2AroBNQC9gL+BSI\nD6x1zKwm0AZ4H3gjmT9ySZcDDwB3AQ2A7YDzgI5AlWi3F4BfgSbA1sBpwG/FnK8K8CHQEjgSqA3s\nD8wH9k5UniQ0Acan4DxlaSKwoTYtKR/4KzA5VReIzulyhZn5lqYNaA8sKiH/FmAckFfCPk0BA/Lj\n0q8gBMeSjv0TsBw4PkE5lwFtk7ynf0TXrVnCPi2AT4BFhCB6dExef+AR4G1gKfAV0CzKmwysB1ZG\nZaoKTAUOjTm+L/Bi9Hs14EVC0F8EfA1sF+V9Avwj+j0P6A1MA34Hngf+FPf+ng5MB+YB15Vwb/2B\nu4E5QN0o7SjgHWA40CtKawZ8FJVtHvAS4cMTwgdd7H1eFVOOs6JyDIv9tyd8cM8AukfnqAlMAk7L\n9P/nviXevMabXhOBdZKek9RVUt24/EOBN8xsfSnOPQDYFti9hH32IwSvNxOcawTwSNQksWOCfQ8F\nhprZsqIyJVUGBgHvReW7GHhJUmw5/06o+dclBI9+AGbWjBB0ulv4ir06QVlOJ3y4NCbU1M8jBLN4\nvaKtC7AzIWg9HLfPAYT38hCgj6QWJVx3FfAW0DN6fRohmMcScBuwPeGDqDHhQwMzO5XC93lnzHEH\nRvsfEXsyM1sAnAk8JWlb4D5grJnFX9dlIQ+8aWRmSwh/0AY8BcyV9Jak7aJd6hNqTgBIOlrSIklL\nJb2X4PSzop/1StinPjDPzP6IucYX0TVWSuocJf8V+Ay4HpgiaWxsO3ScrYHZJVxzX0Jgu93M1pjZ\nR4Tmlr/H7DPAzEZG5XoJaFvC+UqyNirPLma2zsxGR+95vJOBe83sl+gD419Az7iv8zea2UozG0f4\nFtImwbWfB06T9CdCsCzUHm1mk8zsfTNbbWZzgXuj/RLpa2bLzWyTDxAzew94jdDU8xfg3CTO57KA\nB940M7MfzayXme0AtCLUgO6PsucDDWP2fcvM6gCXsrH9tTiNop8LSthnPlA/NsCY2f7RNeYT/f9g\nZgvN7Boza0loAx4LDCx4+FbEORsWkV5ge+DXuFr8tJjyQsyHDbCCEKhL4wXgXeDl6OHknVGNu6gy\nTYsrTz7hXktVJjMbDmxDaMIYHB8oJW0r6WVJMyUtITSJJPPw8dcE+U8S/j961szmJ3E+lwU88GaQ\nmU0gtBG2ipI+BHpIKs2/y7GE9sqfStjnS8KDvGM2o4zzCG2Y21N0bfoD4AhJNYo5xSygcdw97QjM\nTLYMcZYTHj4WaBBT1rVmdqOZ7UF4wHcU4Wt/UWVqEleePyjmAeJmeBG4nE2bGSA0MxjQ2sxqA6cQ\nmh82FL+YcxY7b2vUreyJ6HrnS9qlNIV26eeBN40kNZd0uaQdoteNCV+5R0S73Eto53xBUjMFtSjh\nq7ek7SRdBNwA/Kuk9mEzW0RoS31U0gmSakrKk9QW2BA4Jd0hqZWk/Oj65wOTiqlRFfSAeD26vzxJ\nW0u6VlI3wsOy5cBVkipLOgjoDryczHtWhLGEZoHKktoDJ8SUu4ukP0cBaQmh6aGorlf/BS6VtJOk\nmsCtwCuxTTCl9CCh98mwIvJqER6cLZLUCLgyLv83Qnvz5rg2+nkm4cPxeXkf33LBA296LQX2Ab6S\ntJwQcL8n1JIKapf7Eh7WDI/2H0v4oz0/7lyLonN8B3QD/mpmzyQqQPTg5jLCk/PfCX/wTwBXE7qy\nQahRvkHoGfALoXZ49CYnC+dbTXjANoHQrW0JMJLwNforM1sTHduV8DT/UcKT9wmJylqM6wk9BBYS\nPkT+E5PXAPhfVIYfCd3wXiziHM8QPjCGAVMI7/fFpSzPBma2wMw+NLOiaqk3EroGLib04BgQl38b\n0Dtqb78i0bUktSP8O55moV/vHYTa8TVbcg8uPVT0/yPOOefKitd4nXMuzXw0TI6J+t3+UEz2HmY2\nPZ3lcc5typsanHMuzSpsjVf51U1VamW6GFmpdfPGmS5C1vJ6SvF+nT6NBfPnFdXXu9Qq1W5i9kdR\ngw8Ls5Vz3zWzI1N57bJUcQNvlVpU3f1vmS5GVnrv0/syXYSstXadR97idOuyX8rPaX+sTOrvdNXY\nR1IxE17aVNjA65wrDwSlGk+U3TzwOueyl4C83BsT4oHXOZfdipwipHzzwOucy2Le1OCcc+nnNV7n\nnEsj4TVe55xLL/nDNeecSztvanDOuXTyh2vOOZdewmu8zjmXdl7jdc65dBJU8odrzjmXPt6dzDnn\nMsDbeJ1zLp28V4NzzqWf13idcy6N5CPXnHMu/bypwTnn0sybGpxzLp1y8+Fa7t2Rcy63SIm3pE6j\niySNkrRaUv9i9rlBkkk6NCatqqRnJC2RNEfSZXHHHCJpgqQVkj6W1CRRWTzwOueylwR5+Ym35MwC\nbgGeKfpSagacAMyOy+oL7Ao0AboAV0k6MjqmPjAAuB6oB4wCXklUEA+8zrnslqIar5kNMLOBwPxi\ndnkYuBpYE5d+GnCzmS00sx+Bp4BeUd5xwHgze83MVhGCdBtJzUsqiwde51x2U17iDepHzQgF2zmb\ndQnpr8AaMxsSl14X2B4YF5M8DmgZ/d4yNs/MlgOTY/KL5A/XnHPZLbka7Twza1+606smcCtweBHZ\nNaOfi2PSFgO1YvLnxh0Tm18kD7zOueyltPRquBF4wcymFJG3LPpZG1gV8/vSmPzaccfE5hfJmxqc\nc1lNeXkJty10CPB/UY+FOUBj4FVJV5vZQsLDtjYx+7cBxke/j4/Nk1QDaBaTXyQPvGXk8RtOZtqH\ntzHqtWs3pF13bjcmv3sLI16+hhEvX8MRB+wBQM+u7TekjXj5GpaPfpDWuzUCoO+F3fn5nZuZ+/k9\nGbmPsvbPC85mj50b0XmfthvSxn83jm6HdOLAfffklL/1YOmSJQBMnzaVJtvW5uCO7Tm4Y3uuvOTC\nTBU7LWbN+JW/dj+cg/ZpzcH7teXpxx8C4K5+fTm0YzsO79SBk47rxpzZswCYNHECRx/emZ23q8Xj\nD92byaKnTFiAQgm3pM4l5UuqBlQCKkmqJimfEHhbAW2jbRZwLvBIdOjzQG9JdaOHZmcD/aO8N4BW\nko6Pzt0H+NbMJpRUFg+8ZeSFQSM45sJHNkl/6MWP2bfn7ezb83beHf4DAC+/M2pD2lm9n2farAV8\nO3EmAEOGfUenU+9Ka9nTqefJp/HygMGF0i676Dx639iPT0eMoVv3HjzywMYPnSY77cxHn4/io89H\ncdf9m76/uaRSfj59brmDT776lrfe+4znnn6ciRN+5LyLL+ODz0fz3mdfc8gR3bj/zn4A1Klbj5tu\nv5dzL7o0wyVPISW5Jac3sBK4Bjgl+r23mc03szkFG7AOWGhmBc0MNxAemE0DPgXuMrOhAGY2Fzge\n6AcsBPYBeiYqiAfeMvL5N5NZsHjFZh/3tyPb8erQ0Rtej/xuKnPmLUll0bLKfh07Uadu3UJpkyZN\nZL+OnQA4sMshvP3WG5koWsZt16Ahf26zJwA1a9Vi192aM2f2TGrV3tikuHL5ig01vvrbbEvbvdqT\nX7lyRspbNhLXdpOt8ZpZXzNT3Na3iP2amtkHMa9Xm9mZZlbbzLYzs3vj9v/AzJqbWXUzO8jMpiYq\niwfeNDuvZ2dGvvIvHr/hZOrUqr5J/gmH78WrQ0dloGTZo3mLlgwdMgiAQQNfZ+bMGRvypk+byiEH\ndKBH10MY8cXwTBUx7X6dPpXvvx3Hnu32BuCOm/vQoWUz3njtv1xx7Q0ZLl3ZSlXgzSZZEXglTZW0\nUtKymO3hmPyDomF8V8Ud1zRKLzhmqqRr0n8HyXnqtc/Yo3tf9ul5O3PmLeH2y44rlN+hVRNWrFrL\nD5PjB85ULPc/+iTPPvk4h3Xeh2VLl1KlchUg1AC/GT+ZD4d/zY233sX5Z522of03ly1ftoxzTutJ\n39vu3lDbvfr6m/h6/GSO/evfefapxzJcwrKVl5eXcCtvsqnE3c2sZsx2UUze6cCC6GdR6phZTeDv\nQJ+C4XzZ5vcFS1m/3jAznhnwOe1bFR7S/dcj2lX42i7Arrs159U3h/D+sK849oQTabLTzgBUrVqV\neltvDUCbPfei6U47M3nSz5ksaplbu3Yt55x+Isf+tSfduvfYJL/HCSfyTi43xaS2jTdrZFPgLZKk\nrQjjpy8EdpVUbCdpM/uS0I2jVZqKt1ka1N/YNnfMwW0K1Wwlcdxhe/Lau6OLOrRCmTv3dwDWr1/P\nfXfdxulnhUFI8+bNZd26dQBMnfILv0yeRJOmO2WsnGXNzLji4nPZZbfmnHPhJRvSf5m88cPmvaGD\nabbb7pkoXloohW282aQ8DKA4ntBJ+TXgZMK46U2qhQrv/v6EoXpjijpRNIww/BVXrlnULinz3G29\n6NRuV+rXqcmkoTdz8+ND6NxuV1rvvgNmxrTZC7j4lv9u2P+AvXZh5m+LmDqz8DDyfv88hhO7tmer\napWZNPRmnn3jS/o9MST+cuXWuWecwhfDh7Fg/jzaNt+JK6/tw/JlyzZ8fe52dA/+fkr4ojPi88+4\ns9+NVMrPp1KlStx5/8PUrVcvk8UvU1+P+ILXX3mJ5nu04vBOHYDQxPDyi/355eeJKC+PHRrvyG33\nhla533+bQ7eD92fZ0iXkKY+nH3+Yj78cW+hhXHlUHgNrIjKzTJcBSVOB+sAfMclXmtlTkj4Avjez\nSyT9HXgQ2N7M1kpqCkwhDNEzYA7wmJk9mOiaeVtta1V3/1tqbyRHTBt2X6aLkLXWrsv830u26tZl\nP8aNGZ3SKJm/9c5Wu9stCfdb+OLJo0s7ZDgTsqnG2yO2CweApMaEadj+FSW9CTwJ/AUYGLNrfTOL\nDdrOuVwgUF7u1XizKfAW5VRCO/SgmK8b1QjNDQOLO8g5lztysakh2wPvaYQJLB6PSdsbeE3S1pkp\nknMuXQoeruWabAq8gySti3n9NdAUeCQallfgLUmTCF3HCo81dc7lHA+8ZcTMmm7m/rGTDOfev4pz\nbqMc/AvPisDrnHNFEuVyZFoiHnidc1nNmxqccy6N/OGac85lQu7FXQ+8zrksJm9qcM65tPOHa845\nl265V+H1wOucy2652NSQe3V451zOSGYu3s1YZfgiSaMkrZbUPyZ9X0nvS1ogaa6k1yQ1jMmXpDsk\nzY+2OxVzUUltJY2WtCL62ZYEPPA657JaCidCnwXcAjwTl16XMOthU6AJsBR4Nib/HKAH0AZoDRxF\nWP4dSVUIsya+GJ3nOeDNKL1YHnidc1lNeUq4JcPMBpjZQGB+XPo7ZvaamS0xsxXAw0DHmF1OB+4x\nsxlmNhO4B+gV5R1EaLK9P1qN+EFCq/TBJZXFA69zLqslWeOtHzUjFGznbMElOxOWECvQEhgX83pc\nlFaQ960VXlHi25j8IvnDNedc9kq+H++8VKxAIak10Ac4Jia5JmGVmwKLgZpRO298XkF+rZKu44HX\nOZe1BKSrU4OkXYB3gH+a2WcxWcuA2IXragPLzMwkxecV5C8t6Vre1OCcy2LpWWVYUhPgA+BmM3sh\nLns84cFagTZsbIoYD7RW4UK0pnBTxSY88DrnslpenhJuyZCUL6kaUAmoJKlalNYI+Iiw6MLjRRz6\nPHCZpEaStgcuB/pHeZ8A64D/k1RV0kVR+kcllcWbGpxz2UspbWroDdwQ8/oUwtJiBuwM3CBpQ76Z\n1Yx+fSLK/y56/XSUhpmtkdQjSrsd+JGwcO+akgrigdc5l7UESddoEzGzvkDfYrJvLOE4A66KtqLy\nxwDtNqcsHnidc1ktB0cMe+B1zmW3XJyrwQOvcy5rSalrasgmHnidc1nMl/5xzrm0y8G464HXOZfd\nvMbrnHPplNp+vFnDA69zLmulsh9vNvHA65zLat7U4JxzaZaDcbfiBt7WzRvz3qf3ZboYWemIez9L\nvFMF9cW1XTJdhKxVqSyaBJKfj7dcKTbwStoq2ZNEy2U451xKpXM+3nQqqca7jDBrTzIqpaAszjkX\nJ/lpH8uTkgJv17SVwjnnilGhmhrM7N10FsQ55zaRo/14k16BQtLWki6SdJ+kraO0DpIal13xnHMV\nWWjjLfulf9ItqV4NktoAHwJzgWbAQ4S16Y8GmgKnllH5nHMVXHkMrIkkW+O9B3jazFoAq2PS3wE6\npbxUzjkXSdWaa9kk2X68HYBzi0ifCTRIXXGccy5GBW/jXc2ma8cD7AbMS11xnHNuI6VweffoGdUo\nSasl9Y/LO0TSBEkrJH0cLfdekFdV0jOSlkiaI+myZI8tTrKBdxDQW1JBDdmiJZFvB95I8hzOObfZ\npMRbkmYBtwDPFD6/6gMDgOuBesAo4JWYXfoCuwJNgC7AVZKOTPLYIiUbeK8AdgDmANUJa8b/AqwB\nrk3yHM45t9nypIRbMsxsgJkNJHQMiHUcMN7MXjOzVYRA20ZS8yj/NOBmM1toZj8CTwG9kjy2SEm1\n8ZrZQkn7AUcCexEC9jfAEDNbn8w5nHNuc23Gmmv1JY2Kef2kmT2Z5GVaAuMKXpjZckmTgZaSfgO2\nj82Pfu+R6FhgQnEXTHqSnCjADok255xLiyQ7Lcwzs/alvERNQlfZWIuBWlFewev4vETHFmtzBlAc\nKek9STOi7X1JPqzYOVem0jCAYhmbdh6oDSyN8ojLL8hLdGyxkgq8ki4G3gJ+A26NtjnAQEn/l8w5\nnHOuNFL4cK0444E2G6+nGoSBYuPNbCEwOzY/+n18omNLumCyNd5rgEvN7FQzezTaTgUujfKccy7l\nRNSlLMF/SZ1LypdUjTCbYiVJ1aKeWm8ArSQdH+X3Ab41s4I22ucJvbrqRg/Nzgb6R3mJji1SsoH3\nT0BRk+a8R9H9e51zbstJVMpLvCWpN7CSUFk8Jfq9t5nNBY4H+gELgX2AnjHH3QBMBqYBnwJ3mdlQ\ngCSOLVKyD9cGAccQhg7HOgZ/2OacK0OpGrlmZn0J3b2KyvsAKLILmJmtBs6Mts06tjglrUBxQczL\n7whV7U7AiChtX+BA4M7NuaBzziVLkHQ/3fKkpBrv9XGvVxGq0fvEpV0M3JbicjnnHJCbczWUNBF6\nw3QWxDnnipKL00JW2FWGnXPZTyqj1YszLOnAK6kpYVzyjkCV2Dwzu6CIQ5xzbovlXthNfgWKwwgD\nKCawcWzyzoT+cCPLrHTOuQovF5saku3Heytwu5ntSZib90RCzfdTQkB2zrmUC70aEm/lTbKBtznw\nYvT7H0B1M1tOGKVxZVkUzDnnSGKehvJYI0428C5nY7vubMJYZAADtk51oZxzrkBFXnNtJLA/8CMw\nFLhLUgvCUDlv43XOlYmCpoZck2yN90pgbPT7DcAXwFnA79FPV4J/XnA2e+zciM77tN2QNv67cXQ7\npBMH7rsnp/ytB0uXLCl0zIxfp7NTw7o8+uC96S5umbvh6OZ8eMUBvHb+3hvSDt1jG/53/t6M7tOF\nPRpunMr0T9XzefK0Pfn8X525uutuhc7z1Ol78saF+/DyuR14+dwO1N2qctruIR3OPftMmjTajvZt\n/7whbcGCBRzV9XD+vMduHNX1cBYuXAjAsE8/oUH9OuzTfk/2ab8nt95yU6aKnXIVtqnBzH4ys9HR\n70vN7Awz2w04ltzs7ZFSPU8+jZcHDC6UdtlF59H7xn58OmIM3br34JEHCk+D0edfV3DIYUeks5hp\nM2jsHC58cWyhtMm/L+fyV7/nm2mLCqWv/mM9j378C/e9N6nIc1034Ad6PvE1PZ/4moUr1pZZmTPh\n1NN6MXDwO4XS7rnzdg7qcjDf/TCRg7oczD133r4hb/8DOvHVqDF8NWoM1/buk+7ilhklsZU3SU+E\nXow9gJ9TUZBctl/HTtSpW7dQ2qRJE9mvYycADuxyCG+/tXHN0CGD36RJ053ZvfkeaS1nunwzfRGL\nV/5RKG3KvBVMm79ik31XrV3P2F8Xs/qPirfC1AGdOlOvbr1CaYMHvcXJp54OwMmnns6gt97MRNHS\nRkrdmmvZZEsDryul5i1aMnTIIAAGDXydmTNnALB8+XIevu9urrimdyaLV270PaYFL5/bgbM7N810\nUdLi999/o2HDMJq/YcOGzJ37+4a8kSO+ZJ92bTmmezd+GF/iPNzlSi4+XMuKwCtpqqRD49J6SRoe\nk79S0jJJv0l6VlLNKO8TSf/IRLm3xP2PPsmzTz7OYZ33YdnSpVSpHDqN3HXrTZx74f9Ro2bNBGdw\n1w4Yz98eH8mZz37DnjvW4ajWDTJdpIxpu+deTJg0la9Gj+X8Cy7ixL8em+kipUwaVqBIu6wIvEnq\nbmY1CascdyBMalxu7bpbc159cwjvD/uKY084kSY77QzAN6NGcnOfa2nfaleefOwhHrj7Dv79xKMZ\nLm12mrt0DQAr1qzjne/m0LJR7s/Jv+222zF79mwAZs+ezTbbbAtA7dq1qRl9WB/ZtRtr165l3rx5\nGStnqojEzQzlsamhxO5kkhI1MjZLkJ9yZjZT0jtAq3RfO5Xmzv2dbbbZlvXr13PfXbdx+lnnAPDW\nux9v2OeuW2+iRs2anHWuT4URr5JErWr5LFq5lvw80Xm3+nz1y4JMF6vM/aV7d1564TmuuOoaXnrh\nOY7qfjQAc+bMYbvttkMSX389kvXr17P11jnQxb6c1mgTSdSP93vCIIniKEF+yklqDHQDBpTi2HOA\ncwB2aLxjiktWvHPPOIUvhg9jwfx5tG2+E1de24fly5bx7FOPAdDt6B78/ZTT01aeTLvtuJa0a1qH\nOltVZuil+/P4J1NYvHItV3fdjbpbVeHBk9rw05ylXPjSOADe/ud+1KiaT+VKokvz+lzwwlhmLV7F\nI6e0Ib9SHpUEX01ZyIBvZmX4zlLr9FNOYtiwT5g/bx677NSY3n36cvmV13DqSSfyXP9naNx4R178\n76sAvDHgfzz9xOPk5+dTrXp1nn/xv+Wym1VRcuU+Ysms+LgpafdkTmJmP21RIaSpQH3CcOQCVYBv\nzOyAuPzFwNvA5Wa2UtInwItm9vTmXLPtXu3svU9HJN6xAjri3s8yXYSs9cW1XTJdhKzVcd8OfDN6\nVEqj5Ha7tLIT7/5fwv0eOrbFaDNrn8prl6US23ij/rsJtxSVpYeZ1SnYgPjv1wX5TczsAjNbmaLr\nOueyWKomyZHUVNIQSQslzZH0cLTKMJLaShotaUX0s23McZJ0h6T50XantrAaXp4erjnnKqAUzk72\nKGG0bUOgLWHNyAskVQHeJEwEVhd4DngzSofQPNkDaAO0Bo4Czt2ie9qSg7NIvqRqMVtujR11roIK\n3cVSNmR4J+BVM1tlZnMI8860BA4iPO+638xWm9mDhOdXB0fHnQ7cY2YzzGwmYbX1XltyX7kSeB8D\nVsZsz2a2OM65VEmyxltf0qiY7ZwiTvUA0FPSVpIaAV3ZGHy/tcIPvL6N0mHj4g8FxsXklUpWrLlm\nZk2LSOsP9C8uP2a/g8qmVM65TBNJr7k2L4mHa58CZwNLCKvnPAcMJIwJWBy372KgYLammnH5i4Ga\nkhQXrJO2WTVeSTUltfGv8s65dMlLYktEUh7wLqEbag1CL6m6wB3AMiB+9E1tYGn0e3x+bWBZaYMu\nSZYZSTUkPU/4pBgNNI7SH5Z0XWkv7pxziaRoyHA9Qtx6OGrHnU9okuwGjAdax/VUaB2lE/1sE5PX\nJiavVJKt8d5GWP5nf2BVTPp7wF+3pADOOVccJTFcOJkhw2Y2D5gCnC8pX1IdwkOzccAnwDrg/yRV\nlXRRdNhH0c/ngcskNZK0PXA5UTNoaSUbeI8BLjazERQeqfYDYbVh55wrEymcJOc44EhgLjCJMCDr\nUjNbQ+gudhqwCDiTMG5gTXTcE8Ag4DvCaN63o7RSS/bh2jaE/m/xamzJxZ1zriQC8lM07aOZjSV0\nHSsqbwzQrpg8A66KtpRItsY7mtAWsqEs0c8zgS9TVRjnnIuXi9NCJlvjvQ4YIql5dMyFkgo6Hh9Y\nRmVzzlV0mzcyrdxIds21YYQAuy0wk9BWshzoaGa+yrBzrswoif/Km6QHUESLXZ5YhmVxzrlCcnV5\n96QCr6StSso3s01XKXTOuRRIcuRauZJsjXcZJU94XikFZXHOuUIqdI2XMJlErMrAnsA/gOtTWiLn\nnCtQTnstJJJU4DWzd4tIHixpInAKYWSHc86lXHlczDKRLZ0WchQb56x0zrmUKmhqSNFE6Fmj1NNC\nRrOzX0joXuacc2VAVMrBGm+yvRrmUvjhmoA6wBrC+GbnnEs5UYHbeAkTBcdaT5ho4gszK2oOB+ec\n23LltCkhkYSBN1qFcy0wJFqnyDnn0qZCPlwzsz+Ah4GqZV8c55zbqKCpoaJOkjOSMOv6tDIsi3PO\nbaIij1x7GLgnmn19NGGCnA3M7IdUF8w550TuLIUeK9nA+2r089HoZ0EPB0W/+5Bh51zqKSz/k2uS\nDbwtyrQUzjlXjNwLuwlq8ZKekVTLzH4qaUtXYZ1zFUsYubbli11uOJ/UU9KPkpZLmiypU5R+iKQJ\nklZI+lhSk5hjqkaxcImkOZL60ussAAAZkUlEQVQu29L7StR8cjpQfUsv4pxzpZWqIcOSDgPuAM4A\nagGdgV8k1QcGECb8qkeYCuGVmEP7ArsCTYAuwFWSjtySe0rU1JCLtXznXLmhVLbx3gjcFK2WDtF0\nB5LOAcab2WvR677APEnNzWwCYXTuGWa2EFgo6SmgFzC0tAVJ5oFhSfPwOudcmSno1ZBoS3geqRLQ\nHthG0iRJMyQ9LKk60BIYV7CvmS0HJgMtJdUFto/Nj35vuSX3lczDtTmJPnHMzHs1OOfKRJI13vqS\nRsW8ftLMnox5vR1hHvETgE6E0bhvEqZDqEmYAiHWYkJzRM2Y1/F5pZZM4D0HWLQlF3HOudJKsqFh\nnpm1LyF/ZfTzITObDSDpXkLgHQbUjtu/NrCUsPpOwetVcXmllkzgHZSrE+F4A3bRvrzOp1guTosr\n3850EbLWrBmLE++0mSRSMi2kmS2UNIOim07HEzoSRNdUDaAZod13oaTZhJG770e7tImOKbVEzSPe\nvuucyyhJCbckPQtcLGnbqO32EmAw8AbQStLxkqoBfYBvowdrEFbY6S2prqTmwNlA/y25p0SB1yuF\nzrmMUhJbkm4GvgYmAj8CY4B+ZjYXOB7oBywE9gF6xhx3A+Fh2zTgU+AuMyt1jwZI0NRgZrk4TNo5\nV46kqjeZma0FLoi2+LwPgObFHLcaODPaUqLUS/8451xZC93Jcu+Ltwde51wW27whweWFB17nXFbL\nwbjrgdc5l728qcE559KtnC7tk4gHXudcVvPA65xzaSRSM3It23jgdc5lNXkbr3POpVcOVng98Drn\nspvXeJ1zLo3CmmuZLkXqeeB1zmWvzVzMsrzwwOucy2q5F3Y98DrnsljB8u65xgOvcy6r5V7Y9cDr\nnMt2ORh5PfA657KaNzU451ya5V7Y9cDrnMt2ORh5fU0151zWCotZJv5vs84p7SpplaQXY9JOkjRN\n0nJJAyXVi8mrJ+mNKG+apJO29L488Drnslc0H2+ibTM9QlhtOFxCagk8AZwKbAesAB6N239NlHcy\n8Fh0TKl5U4NzLqul8tmapJ7AIuALYJco+WRgkJkNi/a5HvhRUi1gPWHp91ZmtgwYLuktQpC+prTl\n8Bqvcy6LJdPQIID6kkbFbOdsciapNnATcHlcVktgXMELM5tMqOHuFm3rzGxizP7jomNKzWu8zrms\nlmSNd56ZtU+wz83Av83sVxU+aU1gcdy+i4FawLoS8krNa7xp8M8LzmaPnRvReZ+2G9K++3YsXQ8+\ngC4d23PYgfvyzajQ5PTwA/fQpWN7unRsT+d92tKgTjUWLliQqaKn3cMPPkC7tq3Yq01LHnrg/kJ5\n9917N9Uri3nz5mWodGXvjp6t+fqmQxl6VecNaf/q3pwPrjmQd67sxONntKNWtY31pfMPacbH1x7E\nh/86kM671wegSn4eAy/pyJArOvHu1Z255Mhd034fqaIkt4TnkdoChwL3FZG9DKgdl1YbWJogr9Q8\n8KZBz5NP4+UBgwul3XT9tVxxTW8+/nwUV197Azf1+RcAF/3zcj7+fBQffz6K6/rewv4HdKZuvXpF\nnTbnjP/+e5595ik++2IkI0eP450hg5n0888A/Prrr3z0wfs03nHHDJeybL0+cga9nhxZKG34xHkc\ncecwut71GVPmLueCQ0PT5C7b1aT7nttzxB3DOP2Jkdx0QivyBGv+WM9Jj46g292f8Ze7PuPA5tvQ\ntkmdTNxOaqQi8sJBQFNguqQ5wBXA8ZK+AcYDbTZcTtoZqApMjLZ8SbGfXm2iY0rNA28a7NexE3Xq\n1i2UJomlS5cAsGTJYho0aLjJcW+89grHnnBiWsqYDSZM+JG9996Xrbbaivz8fDp1PpA333wDgKuu\nuJR+t92JcnAUU6yRvyxg0fK1hdI++2ke69YbAGOmLaRBnWoAHNZqOwaNmcWadeuZsWAl0+atoM2O\nIcCuWLMOgPxKIr9SHlgabyLF8qKpIUvakvAk0AxoG22PA28DRwAvAd0ldZJUg9AOPMDMlprZcmAA\ncJOkGpI6AscAL2zJPXkbb4bccsfdnHjsUfTtfQ3r16/n7fc/LZS/YsUKPvrgPW67+4EMlTD9WrZs\nRd8+1zF//nyqV6/O0HeGsFe79gwe9Bbbb9+I1m3aJD5JjvvbPo0ZPGY2AA3+VI0x0xZtyJu9aFUI\nytPC5OGDLj+AJvVr8MLwaYydvqi4U2a9VHzUmtkKQjexcE5pGbDKzOYCcyWdRwjAWwMfAGfEHH4B\n8AzwOzAfON/MtqjGm7bAK2kqsD2wvZnNi0kfS6i67w+8C9Q1s3VR3lPAiUWkrTOz86K+dPcBHQj/\nPpOB681sSLruq7T6P/0kN912F92POY43B7zGJRedy+tvDd2Q/947g9l73/0qTDMDQPMWLbj8iqs5\n6sjDqFGzJq1btyE/P587buvH4Hfey3TxMu7CQ3fhj3XGwNEzgaIfOllUs11v8Je7h1OrWj5PnNme\n3RrUZOKcZWksbYok35SwWcysb9zr/wD/KWbfBUCPVF4/3U0NU4C/F7yQ9GegevRyHlAJ2Ctm/07A\nrLi0zsCw6PdBwPuEjs3bAv8HLCmLgqfaK/99gaOOPhaAo489gTGjvy6U/8brr1aoZoYCvc48iy+/\n/oYPPh5G3Xr1aNKkKdOmTmHvdm3YfZemzJwxg/323os5c+ZkuqhpdVyHRhzcclsueXHMhrTZi1bR\nMGp2AGhYpxq/LV5V6Lilq/5gxOT5HNh827SVNdVSPXItG6Q78L4AnBbz+nTg+ej3tcAIQmBF0rZA\nFeCVuLTdgGGS6gM7AU+Z2Zpo+9zMhqflTrZQgwYN+WJ4+Pz47NOP2bnZLhvylixezJfDP+PIvxyd\nqeJlzO+//w7A9OnTeXPgAE4+9TSmz/qdnyZN5adJU2m0ww58OfIbGjRokOGSpk/n5ttw3sHNOPvp\nUaxau35D+gfjf6P7nttTpVIeO9SrTtNtajBu+iLq1aiyoedD1cp5HLBbfSb/Xg5ru0QV3tSPXMu4\ndLfxjgBOldSC8LTwROAA4JYofxghyN4T/RwebRfFpE0xsxkKT1kmAS9Kehr40sx+K+niUafqcwB2\naJy+p+PnnnEKnw8fxoL582jTfCeuurYP9zz0OL2vvow//viDalWrcc8Dj23Yf8jgNzno4EOpUaNG\n2sqYLf7+t+NZsGA+lfMrc/+Dj1A37qFkrnvg1Lbsu8vW1K1RhS9uOJj7h/7M+Yc0o0p+Hi+cvzcA\nY6Ytovdr3/PznGW8PXY2713TmXXrjT7/+571BtvWrsrdJ7WhUp6QxNtjZ/HRD79n+M5KrzwG1kRk\nlp7HnVEb7z+AfYEawKeEESRdCbXdnQjdPV4H6gP3A98D/yW03TaI0mqb2RnROXcgDNvrGh0/HDjL\nzH5OVJ62e7Wz9z8dkbL7yyW1qlfOdBGyVosr3850EbLWrP9cwurffk5pmGzVZi/739DEX2JbbF9j\ndBIDKLJGJrqTvQCcBPRiYzNDgRGEUSStCLXbz6Lx0b/GpBW072JmM8zsIjNrBjQBlhdxTudcOZaL\nTQ1pD7xmNo3wkK0boX9cbN4qwqxBRwENzWxClPVZlNaamMAbd+yvhFmEWpVNyZ1zmZCa8RPZJVMD\nKM4CDo46J8cbBlxCmD2owPAobU40gQWS6kq6UdIukvKih21nEmrNzrlckYORNyOB18wmm9moYrI/\nJXQNi23YGR6lxdZ21xDahD8gdCH7HlhNaMJwzuUAKWUj17JK2no1mFnTYtL/IOYzy8zeJe4zLOqt\nEJ+2nNAdzTmXw8pfWE3Mhww757JbDkZeD7zOuSxWPkemJeKB1zmX1cphE25CHnidc1mrYMhwrvHA\n65zLat7U4JxzaeY1XuecS7McjLseeJ1zWUzk5HJPHnidc1nLH64551wG5GDc9VWGnXPZLRXTQkqq\nKunfkqZJWippjKSuMfmHSJogaYWkjyU1iTv2GUlLJM2RdNmW3pMHXudcVkvRmmv5hHm9DwT+BFwP\nvCqpaTSz4YAorR4wirDkWIG+wK6EOb+7AFdJOnJL7smbGpxzWS0VbbzRpFp9Y5IGS5oCtCMs6T7e\nzF4L11NfYJ6k5tGc4KcBZ5jZQmBhtNJ5L2AopeQ1Xudc1kqmmSEKzPUljYrZzin5vNqOsHDueKAl\nMK4gLwrSk4GWkuoC28fmR7+33JL78hqvcy6rJdmUMC/ZNdckVQZeAp4zswmSagJz43ZbDNQiLEVW\n8Do+r9S8xuucy24pXIFCUh5h3cc1hNXLAZYBteN2rQ0sjfKIyy/IKzUPvM65rJaquKswEuPfwHbA\n8Wa2NsoaD7SJ2a8G0IzQ7rsQmB2bH/0+vrT3Ax54nXNZLfGyP5ux9M9jQAugu5mtjEl/A2gl6XhJ\n1YA+wLcxi+0+D/SO1nlsDpwN9N+Su/LA65zLWgUj11LQj7cJcC7QFpgjaVm0nWxmc4HjgX7AQmAf\noGfM4TcQHrZNI6wJeZeZlbpHA/jDNedcBWBm0yihVcLMPgCaF5O3mrCC+ZmpKo8HXudcVvO5Gpxz\nLs18InTnnEsjCfJyL+564HXOZTkPvM45l17e1OCcc2nmD9eccy7NcjDueuB1zmU3X3PNOefSKFfX\nXJOZZboMGSFpLmEIYDaoD8zLdCGylL83Jcum96eJmW2TyhNKGkq4x0TmmdkWrQqRThU28GYTSaOS\nnUu0ovH3pmT+/pRPPkmOc86lmQde55xLMw+82eHJTBcgi/l7UzJ/f8ohb+N1zrk08xqvc86lmQde\n55xLMw+8zjmXZh54s5BycYykc24DD7xZyPyJpyslSd2jlXBdFvO5GrKIpLOBBsBXwCgzW5DhIrly\nRNIrwDZAP0lTokUaXRby7mRZQtJAoBGwDNgWeMDMnpQkrwG7RKKguz3wF2Clma2Ny88zs/UZKZzb\nhDc1ZAFJg4DqZtbBzLoAnwLnSsqv6EFX0gmSWma6HNlMUntgR+BIM1tiZmslbSPpQEmHAZjZekn+\n954lvKkhwyR1IdRSDopJfpNQ+60BLM5AsbKCpFbAzcBbkv5tZhMzXaYsZUAtM1suaVtgL+B5YCFQ\nV9IwMzvBa7zZwz8BM8zMPgZuAN6V1FZSZeAp4CMzq7BBF8DMvgdOA/YFzpK0W0GepEoxv/8pA8XL\nOEmXSjrTzEYD1SR9CzwL3Ac8BBwInATsXVDzddnBa7wZVNDuZmY3R18DRwErgWvN7KFonwrXxivp\nVqAZMBe4H/gnIZCYpGfN7CczWxft+0+gjaTzzGxNxgqdZpJqAPsQKk/PEGq5twDjgClm9lG03wrC\nvNNLM1RUVwR/uJYBktoBE81saVz6xcADQCsz+0FS5fiHJLlO0puEia8/BNoSAksPQu3tBuBL4Bkz\n+0nSFcCdQDszG5OhImeMpGbAaOAuM+tXzD7nApcAh5rZzHSWzxXPa7xpJukW4FpguKR+wLdmNhvA\nzB6StA0wVtIRZvZxRXoaLel/QKOCib0l9QBuBxqa2YeSDOgLLIraMs8DOlSkoCvpPOBpIM/MJks6\nH+gr6cuCWm603wFAR+BK4HAPutnFA2/6fQ9MARYA/YExkj4CHgZWm1kfSeuADyV1NrPhmStq+kiq\nBxwBPCBpKzNbAfwArABqRk0uH0WD+h4kPHw8IGrfrBAkHQc8Smjz/kDSMOA1wvvWVdJoM1scNUN0\njfbrYmbfZazQrkje1JAmkqqZ2SpJOwMfA1cA3wCnAL0I7ZnvEL42LpN0EfChmf2YqTKni6Q+hCaE\nGcBbhA+kd4CBwMNmdmfc/u0Ja2xNTW9JM0tSY8J7spwQcLsSarQ7ED64e5nZ59G+1QldFH0QThby\nXg1pIOkp4GQAM/sFuJvQNrnezG4kBOF2wAnAbEmXAY9VkKBbF6gO9APqAccAZwJfAC8VBN3YPqhm\nNqqiBV0AM/sVuBioBPwCvAoMAqoAs4BHJNWK9l3pQTd7eeAtY9HDoj3N7N8xyR8DvwE7SupM6P5z\nAfBnQvvvwIKn9rnOzBYCjxD6Lt8L1AUOB2YS2nLrRftViHbueJKul/SlpF2iJpgvgMHAHmbWH7gI\nOJSw0nBrwv9HLst5U0MZkvQGsI2ZHRCTVsXM1kh6FDiVMET4OjN7JlPlzIRoVN4fMa8rA7cCRwFn\nAEsItbn/EoZPz81IQTNI0tWEUYz9gMrAB8BjhPk8bgcuNbOJknYEdiV0JzvDzCZkqMguSR54y4ik\npwkPPZrG9DndmfBg6CJgNfA+MMzMLqhI/XUl3UZ4yPg/YI2ZmaTtCW3e3wO1gMsJtbgvCTXhfhWp\n1hu18T8IvAecA3QHDgFaAD0JA0v2Bw4s6L9ckXrAlHfe1FB2XiA8kT8cQNIOwDBgbNQ+uQgYQ6i9\nVLSpID8G+gDdo6DbmDAj2wOED6uBhDbwhoRBAv+tgAHlbeAzYG9C08LbhO5zbxOC8QJCs8x5BfM3\nV8D3qNzywFtGzOxTQk3l3qjv5ZfAo2bWO6rdriQEmy6S6lSkyc/N7D3gbODm6L35AnjczG6Lvh38\nB/gI6A38amaTMlfa9IuGQ88iBNnTCR/Qw4AaZnYF4RtTC2A34DigWoaK6krJmxrKWPTwbAgw2Mx6\nxqRfSvi62NPMfspU+TKpqPem4Oty1PSw2szmZ7SQaSRpP2BkTNPUGcClhFrvE4SHaAeY2RRJjQj9\ndL+vqP//lGceeNNAUkfCaKPrzGxAVMu7jTCMs8IMAChK9N48BVwHvBsNnKhwJPUnfBA/TejxckP0\nAXQ7MNvMHlCYs7k1YSRahfoWkGs88KaJpAMJE72MJUwDeZiZfZPZUmWH6L15lPBU/vWKNNlNAUn7\nE9punwd2AgS8S2jnrmJml0T7vU94LrBnbK8QV7544E2jaO7d/xAmrB6X6fJkE0mHEr4FHBw/eVBF\nEdX+HwcuI0xsXovQzl0POMLM3o/228HMZmSsoG6LeeBNs4Khw5kuRzaKmaOhwoo+nB8E+pjZG5La\nArsQhlCv9J4LucEDr3NZRtJBhHbvf5nZ/zJcHFcGfHYy57KMmX0i6R/AY9GIvgrZ7p3LvMbrXJby\ndu/c5YHXuSzm7d65yQOvc86lmQ8Zds65NPPA65xzaeaB1znn0swDr3POpZkHXuecSzMPvK7UJH0v\nqW/M66mSrshAOdpLMklNy/AafSV9n4LzZOQ9ctnFA28OkdQ/CkAmaa2kXyTdLalGmorQgTDLWEKS\neklaVsblib3eJ5IeTtf1nCuJDxnOPR8QFtGsDHQizO9aAzi/qJ0lVTaztam4cEVckNK50vAab+5Z\nbWZzzOxXM/sP8BLQA8LkK1FtuJukkZLWENY4Q1J3SaMlrZI0RVI/SVUKTippW0lvSlopaZqkM+Mv\nHP81WlJtSY9Jmh2d90dJJ0aTwDwL1IipofeNjqki6Q5JMyQtl/S1pCPirnOkpAnROT8jLIGzRSTd\nLumn6P6mSrpT0iZL6kj6h6Tp0X4DJdWPyz9D0g9R2SZKulSS/525QrzGm/tWEmq/se4grOI7CVga\nBbaXgH8S1vbakTAvbFWgIJD2B5oQlp9ZAdwHNC3uotEacu8QFmQ8A5gI7E5YH+wL4BLCcu7NokMK\nmh2ejdJOAmYA3YBBkjqY2bhoYcyBhNm7HiGsyHBv8m9HsZYDZwIzgT0I978auD5mn6bAKcAxwFbA\nk8AzwNHRPZ8N3ARcDIwGWkXlXAt4M4fbyMx8y5GNEBwHx7zem7BE+ivR64MAA46PO24YcH1cWg9C\nMBShRmlAx5j8JsA6oG9M2lTgiuj3w4D1QItiytoLWBaX1iw6Zse49IGEhUIhBOuJRMPdo7TeUfma\nlvDefAI8vBnv5XnApJjXfaP73TEm7YDourtGr6cDp8ad5xLgh6LeI98q7uY13txzZPTQKp9Q032T\nUAOLNSrudTtgb0lXx6TlAdUJy8y0IATEkQWZZjZN0qwSyrEnYa2wHzej7HsRAv0PcYsuVyWsOkxU\nlhFmFjvJyJebcY0iSTqBECR3AWoClaIt1kwzmx7z+iuiDxdJi4DGwBOSHovZJ59wT85t4IE39wwj\nLCu/FphlRT84Wx73Og+4EXitiH3nUrrAUZpj8gg1yA6E8sdauQXnLZGkfYGXCe/BpcAiQvPB3Ztx\nmoJ23ILl6p0rlgfe3LPCNn8F2m+A5sUdJ+lHQmDpQBRUJO0IbJ/gnA0ltSim1ruGTWuUYwiBtYGZ\nfVzMeX8AjpekmFrvviWUIxkdCbXZmwsSJDUpYr9Gkhqb2a/R670J78uPZvabpJlAMzN7fgvL43Kc\nB14H4YHQYEnTgFeBPwgPhvY2s6vM7CdJQwlfo88h1D7vZWMttCgfEr6Kvy7pUkK77C5ADTMbSGjr\nrCbpMELAXWFmEyW9BPSXdDkheNcjtE3/YmYDCA+9Lgful/Qo8GdCLTMZ9aM1zGL9HpWtkaSTCc0W\nRwB/L+L4lcBzki4jNMM8DrxtZj9H+X2Bh6JmhyGEpp69gEZmdluSZXQVgHdzcZjZu4Ql57sQ2nFH\nAtcQHhYV6AVMIbS1DiKsljy1hHOuB7oCnwMvAj8CDwBVovwvCIHrv4TmjKuiQ88g9Gy4E5gADAY6\nA9Oi46YDxwFHAuMITQPXJHmrJxKCfOx2mZkNAu4C7ge+JTwY7FPE8VMJTRKDovfhl6i8Bff8NKFn\nxKlR2T4jNPtMSbJ8roLwidCdcy7NvMbrnHNp5oHXOefSzAOvc86lmQde55xLMw+8zjmXZh54nXMu\nzTzwOudcmnngdc65NPt/8Bi4WStdzqIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2a39b9f8ba8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Confusion Matrix\n",
    "conf_mx = confusion_matrix(y_test, y_pred)\n",
    "plot_conf_matrix(conf_mx, train.author.unique(), title=\"SGD_GS Confusion Matrix\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### LinearSVC"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 338,
   "metadata": {},
   "outputs": [],
   "source": [
    "#SVC\n",
    "#clf__C: 10\n",
    "#clf__tol: 0.01\n",
    "#tfidf__norm: 'l2'\n",
    "#tfidf__use_idf: True\n",
    "#vect__max_df: 0.75\n",
    "#vect__max_features: None\n",
    "#vect__ngram_range: (1, 2)\n",
    "\n",
    "svc_clf = Pipeline([\n",
    "    ('vect', CountVectorizer(stop_words='english', max_df=0.75, max_features=None, ngram_range=(1, 2))),\n",
    "    ('tfidf', TfidfTransformer(norm='l2', use_idf=True)),\n",
    "    ('clf', LinearSVC(C=10, tol=.01))\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 339,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Pipeline(memory=None,\n",
       "     steps=[('vect', CountVectorizer(analyzer='word', binary=False, decode_error='strict',\n",
       "        dtype=<class 'numpy.int64'>, encoding='utf-8', input='content',\n",
       "        lowercase=True, max_df=0.75, max_features=None, min_df=1,\n",
       "        ngram_range=(1, 2), preprocessor=None, stop_words='english',\n",
       "       ... max_iter=1000,\n",
       "     multi_class='ovr', penalty='l2', random_state=None, tol=0.01,\n",
       "     verbose=0))])"
      ]
     },
     "execution_count": 339,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "svc_clf.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 340,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = svc_clf.predict(X_test)\n",
    "y_pred_prob = svc_clf.decision_function(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 341,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.4353762 ,  0.29434548,  0.49251073],\n",
       "       [ 0.56797108,  0.31440796,  0.35121762],\n",
       "       [ 0.40559099,  0.37706917,  0.44071232],\n",
       "       ..., \n",
       "       [ 0.60843503,  0.24479689,  0.35994589],\n",
       "       [ 0.26461942,  0.65582368,  0.28677993],\n",
       "       [ 0.35962621,  0.28986402,  0.56813929]])"
      ]
     },
     "execution_count": 341,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_pred_prob = (y_pred_prob - y_pred_prob.min()) / (y_pred_prob.max() - y_pred_prob.min())\n",
    "y_pred_prob\n",
    "#create_submission_file(y_pred_prob, 'SVC_GS_DF')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 342,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy score: 0.80531154239\n",
      "Log loss: 0.84140525687\n"
     ]
    }
   ],
   "source": [
    "print(\"Accuracy score:\", accuracy_score(y_test, y_pred))\n",
    "print(\"Log loss:\", log_loss(y_test, y_pred_prob))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 343,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV4AAAEqCAYAAABdmDjNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XeYU9XWx/HvbxiKUhQEFZGiAoKg\noGLHig0rtlesYMPe67Vir9eKvaHoteAFFMWuNDtIuaKIIKA0pffOev/YZyDEmUmYySSZzPr4nIdk\n71P2CbKys88uMjOcc86lT16mC+CccxWNB17nnEszD7zOOZdmHnidcy7NPPA651yaeeB1zrk088Dr\nnHNp5oHXZT1JG0nqL2m+pN6lOM9pkj5JZdkyQdKHkrpkuhyu5DzwppmkSZJWSKoblz5SkklqEr3v\nGb3fPWafppIs5v1AScskLYqC0mBJOyZZjmaS3pQ0U9ICSb9JekLS1jH73ChpYnT+KZLeSnDOUyUN\ni/afHgWI9sl9MsU6EdgC2MzMTirpSczsdTM7NAXlWY+kA6K/qz5x6W2i9IFJnqe7pNcS7WdmHc3s\nlRIW12UBD7yZMRE4peBNFCw3KmS/OcBdCc51iZnVADYDBgK9El1cUlPgO2AasLOZ1QL2ASYA7aN9\nugBnAAdH528HfF7MOa8CHgXuIQTJRsBTwLGJypOExsA4M1uVgnOVlZnA3pI2i0nrAoxL1QUU+L/Z\nXGBmvqVxAyYBNwM/xKQ9BNwEGNAkSusJPAzMAPaP0pqGv7K1xw0Ezo15vwOwIokyvAb0T7BPD+DR\nJO9pE2ARcFIx+1QlBOZp0fYoUDXKOwCYAlwN/A1MB86K8m4HVgAro2ucA3QHXos5d5Pos8uP3ncF\nfgcWEr7kTotJHxpz3N7AD8D86M+94z7bO4GvovN8AtQt4t4Kyv8McHGUVilKuxUYGLPvY8CfwAJg\nOLBvlH543H2OiinH3VE5lkb/D6z9eweeBt6JOf/9hC9IZfr/dd+K3vzbMzO+BWpJaimpEnAyIRjG\nW0KoQd6d6ISSqgCnRedO5GDgv0mU8UxJ10pqF5WzKHsB1YC+xexzE7An0BZoA+xO+AIqsCUhgDcg\nBNcnJdU2s9sIn8FbZlbDzF4srtCSqgOPAx3NrCYhuI4sZL86wAfRvpsRvuQ+iKuxngqcBWwOVAGu\nKe7awKvAmdHrw4AxhC+ZWD8QPoM6wH+A3pKqmdlHcffZJuaYM4BuQE1gctz5rgZ2ktRV0r6Ez66L\nRVHYZScPvJnTi/CP9BBgLDC1iP2eBRpJ6lhE/uOS5hFqSZcQaoiJ1CXUpAGQdImkeVHb7PMAZvYa\ncCkhgAwC/pZ0QxHn2wyYZcU3BZwG3GFmf5vZzKicZ8Tkr4zyV5rZgOh+tk/iXgqzBmgtaSMzm25m\nYwrZ50jgNzPrZWarzOwNwt/D0TH7vGxm48xsKfA2IWAWycy+BupI2p7wd/tqIfu8Zmazo2v+m/BL\nINF99jSzMdExK+POtwQ4nfDF8RpwqZlNSXA+l2EeeDOnF6FG1ZVC/oEWMLPlhJ+8dwIqZJfLzGxT\nQo3zKOAdSTsluPZsoH7MNXpE53gUqByT/rqZHQxsClwA3CHpsCLOV1dSfjHX3Ir1a2uTo7S154gL\n3EuAGgnu4x/MbDHhF8QFwHRJH0hqkUR5CsrUIOb9jJjXyZanF+EL8EAK+QUg6WpJv0QPQ+cRavl1\n4/eL82dxmWb2PaFpRYQvCJflPPBmiJlNJrQ/HgH0SbD7y4R/oMcVc741ZjYEGA8kenL/OXD8BpR1\npZn1BkYDrQvZ5RtgGdCpmNNMIzwkK9CIf/4MT9ZiYOOY91vGZprZx2Z2COHLZSzwfBLlKShTUb88\nktULuAgYENVG14qaAq4H/g+oHX3ZzWfdF2pRzQPFNhtIuphQc54GXFfyort08cCbWecAB0W1tCJF\nNcHuhH+0RZK0F+EBW2E/rWN1B/aV9LCkBtGxdYGWMefqKulISTUl5UVNHa0IvSHiyzef8BDpSUmd\nJG0sqbKkjpIeiHZ7A7hZUr3oWrdSeLt2MkYC+0lqJGkT4F8x5d5C0jFRW+9yQpPF6kLOMQBoHnWB\ny5d0MuGze7+EZQLAzCYC+xPatOPVBFYRekDkS7oVqBWT/xfQZEN6LkhqTuj5cjqh6eY6ScU2ibjM\n88CbQWY2wcyGJbn7G4Sn/fF6RG2ziwi1rZvN7MME1x1HeNC1NTBK0kLCU/NpwC3RbguAG4E/gHnA\nA8CFZja0iHM+DFxFeGA2k/Dz+BKgX7TLXcAwQq35f8CPJO4qV1T5PwXeis41nPWDZR7hgdM0Qne8\n/Qk10PhzzCY0zVxNaCq5DjjKzGaVpExx5x5qZoXV5j8GPiR0MZtM+JUQ24xQMDhktqQfE10natp5\nDbjfzEaZ2W+Ev7NekqqW5h5c2ZI//HTOufTyGq9zzqWZB94cFQ3XXVTIdmOmy+ZcRedNDc45l2bF\n9bvMacrfyFSlZqaLkZV23L5hpouQtbyaUrQpf0xmzuxZhfU1L7FKtRqbrVqacD9bOvNjMzs8ldcu\nSxU38FapSdXt/y/TxchKnwx6JNNFyForV6/JdBGy1hEH7Z3yc9qqpUn9O1028slEg1CySoUNvM65\n8kCQgxOyeeB1zmUvAXnFzc9UPnngdc5lN6W02TgreOB1zmUxb2pwzrn08xqvc86lkfAar3POpZf8\n4ZpzzqWdNzU451w6+cM155xLL+E1XuecSzuv8TrnXDoJKvnDNeecSx/vTuaccxngbbzOOZdO3qvB\nOefSz2u8zjmXRvKRa845l37e1OCcc2nmTQ3OOZdOuflwLffuyDmXW6TEW1Kn0SWShklaLqlnEfvc\nJskkHRyTVlXSS5IWSJoh6aq4YzpIGitpiaQvJTVOVBYPvM657CVBXn7iLTnTgLuAlwq/lLYDTgSm\nx2V1B5oBjYEDgeskHR4dUxfoA9wC1AGGAW8lKogHXudcdktRjdfM+phZP2B2Ebv0AK4HVsSlnwnc\naWZzzewX4Hmga5R3PDDGzHqb2TJCkG4jqUVxZfHA65zLbspLvJX2EtJJwAozGxCXXhvYChgVkzwK\naBW9bhWbZ2aLgQkx+YXyh2vOueyWXI22rqRhMe+fM7Pnkju9agD3AIcWkl0j+nN+TNp8oGZM/sy4\nY2LzC+WB1zmXvZR0r4ZZZtauhFe5HehlZhMLyVsU/VkLWBbzemFMfq24Y2LzC+VNDc65rKa8vIRb\nKXUALot6LMwAGgJvS7rezOYSHra1idm/DTAmej0mNk9SdWC7mPxCeeAtI8/cdhqTP7+XYb1vXJt2\n0/lHMOHju/j2zRv49s0bOKz9DgB07thubdq3b97A4uGPs1PzBgBUzq9Ej5tPYXS/WxnZ52Y6dWib\nkfspK1dcfB6ttmvA/nuuu6+fRo/kiA7t6dC+HYfuvyc/Dv8BgHlz53LWaSdy4N67cPiBe/PLzz9l\nqthpMW3Kn/zfMYdy4B5t6LDXzrz4TA8AHry7O4e0b8dh++3OqccfyYzp0wDo2/sNDmnfjkPat6PT\nYQfw80+jM1n8lAgLUCjhltS5pHxJ1YBKQCVJ1STlEwJva6BttE0DzgeejA59FbhZUu3oodl5QM8o\nry/QWtIJ0blvBUab2dhiy2JmyX8KOSRv482t6vb/V2bn32eX7Vi8ZDkv3Hkm7U66BwiBd/GS5Tza\n6/Mij2vVdCt6P9KNHY7uDsDNFxxBpbw8bn/qfSRRZ5ONmT1vcZmVG2DSoEfK9PyxvvlqCNWr1+DS\nC85i0LcjATi50xF0u/gyOhxyOJ998iFPPvZv+n7wGbfffAPVa1Tnmhtu4bdxY/nX1ZfzTv+P01ZW\ngJWr16TtWn/NmM7ff81gxzY7s2jhQo44aC9e6NWb+ls1oGat8Ov2pWef5Ldff+Heh3sw7LtvaLp9\nCzbdtDZffvoxD99/F/0/G5K28h5x0N6MHjE8pcPMKtVpYtU63JZwvyXvnD08UVODpO5A/MluN7Pu\ncftNAs41s8+i91WBpwldzZYC95vZwzH7H0zoEdEY+A7oamaTiiuLt/GWka9+nECj+nU2+Lj/O3xX\n3v5o+Nr3XY7dizbH3QmAmZV50E23vfbZlz8mT1ovTRILFywAYOGC+Wy5ZX0Axv36C5dddR0AzZq3\n4M8/JjPz77+ot/kWaS1zumyxZX22iO69Rs2aNG3eghnTp9K8Rcu1+yxZsnjtw6d2e+y1Nn3n3XZn\n+vSp6S1wmUi+RptIFGC7J7Ffk7j3y4Gzo62w/T8Diu0+Fs8Db5pd0Hk/Tj1qd378+Q9ueLgP8xYu\nXS//xEN34aQrw8PYTWpsBMBtFx/Fvrs2Y+KUmVx5X2/+nlNsu325d8d9D3HK8Udxxy03sGbNGvp/\nMgiAVq13ZED/fuyx1z78OPwHpvw5mWlTp+Zs4I315x+TGDN6JDvvujsA9991K/9983Vq1tqEt9/7\nZ63/zV49ObBDYQ/py59UBd5skhVtvJImSVoqaVHM1iMm/4BoGN91ccc1idILjpkk6Yb030Fynu89\nhB2O7s4ene9jxqwF3HfV8evl79a6MUuWreTnCWHgTH5+HltvWZtvRv7O3qfez3ejJ3Hvlcdlouhp\n9cqLz3H7PQ/y48+/c/s9D3LVJecDcOmV1zFv3lw6tG/HS88+Seud2pKfn3tTBsZbvGgR53c5he73\nPLS2ieH6m+/g+58mcNxJnen5/NPr7f/1kIG89VpPbux+dyaKm3J5eXkJt/Imm0p8tJnViNkuicnr\nAsyJ/izMpmZWAzgFuLVgOF+2+XvOQtasMcyMl/p8RbvW6w/pPumwXXn7o3VdEWfPW8zipct594vQ\nP7vPpz/StmXDtJY5E95+oxdHHhO+YI457kRG/BgertWsVYvHnnqBz4cO44lnX2b27Fk0arxNJota\n5lauXEm3Lp3pdGJnOh7d6R/5nU48mQH9+619/8uY/3Ht5Rfy4uvvULvOZuksatlQkls5k02Bt1CS\nNiY0al8MNJNUZAO6mX1D6MbROk3F2yBb1l3X3e/Yg9qsrdlC+Dl1/CE70/vj4esdM2DwT+zXrhkA\nB+y+PWN/jx9Gnnu23LI+Xw8dDMDQQV+y7bZNAZg/bx4rVoTRnK+/8hJ77t1+bQ0wF5kZ1152Ps2a\nt6DbxZevTZ84Yfza159++AFNm20PwNQpf3DemSfz2NMvsW3TZmkvb1kQiXs0lMemiPLQxnsCoZNy\nb+A0wrjpYfE7KXz6exOG6o0o7ESSugHdAKhco7BdUuaVe7uy767NqLtpDcZ/dCd3PjOA/XZtxk7b\nb42ZMXn6HC696421+7ffpSlT/5rHpKnrDyO/+bF+vHhXFx685gRmzV3E+d1fK9Nyp9sFZ5/O10MH\nM2f2LHZuuQ3X/utWHnr8GW65/ipWrV5F1arVePCx8FP6t3FjufT8s6lUKY/m27fk4R5JDUwqt374\n7mv++9Z/aLFDaw7bL7TtXn/LHbzZqycTxo8jLy+PrRs24p5/PwHAow/cw7w5c7jp2hCkK+XnM+CL\nrzNW/lQpj4E1kazoThZ136gLrIpJvtbMnpf0GfCTmV0h6RTgcWArM1spqQkwkTBEz4AZwNNm9nii\na5Z1d7LyLJ3dycqbdHYnK2/KojtZ/mbbWq0j7kq439zXTkvYnSybZFONt1NBv7kCkhoSpmH7V5T0\nLvAccCTQL2bXumYWG7Sdc7lAoLzcq/FmU+AtzBmEduj+MT83qhGaG/oVdZBzLnfkYlNDtgfeMwkT\nWDwTk7Y70FtSDjyydc4VRykcQJFNsinw9pe0Oub9D0AT4Ekzi5127T1J4wldx95PY/mccxnggbeM\nxA/RS2L/2EmGc+9vxTm3Tg7+C8+KwOucc4US5XJkWiIeeJ1zWc2bGpxzLo384ZpzzmVC7sVdD7zO\nuSwmb2pwzrm084drzjmXbrlX4fXA65zLbrnY1JB7dXjnXM5IZi7eDVhl+BJJwyQtl9QzJn1PSZ9K\nmiNppqTekurH5EvS/ZJmR9sDirmopLaShktaEv2ZcClwD7zOuayWwonQpwF3AS/FpdcmzHrYhLBS\n8ELg5Zj8bkAnoA2wE3AUYfl3JFUhzJr4WnSeV4B3o/QieeB1zmU15Snhlgwz62Nm/YDZcekfmllv\nM1tgZksIS7XvE7NLF+DfZjbFzKYC/wa6RnkHEJpsHzWz5dFc4AIOKq4sHnidc1ktyRpv3agZoWDr\nVopL7kdYQqxAK2BUzPtRUVpB3mhbf0WJ0TH5hfKHa8657JV8P95ZqViBQtJOwK3AsTHJNQir3BSY\nD9SI2nnj8wryaxZ3HQ+8zrmsJSBdnRokNQU+BC43syExWYuA2FVVawGLzMwkxecV5C8s7lre1OCc\ny2LpWWVYUmPgM+BOM+sVlz2G8GCtQBvWNUWMAXbS+oXYifWbKv7BA69zLqvl5SnhlgxJ+ZKqAZWA\nSpKqRWkNgC8Iiy48U8ihrwJXSWogaSvgaqBnlDcQWA1cJqmqpEui9C+KK4s3NTjnspdS2tRwM3Bb\nzPvTCUuLGbAtcJuktflmViN6+WyU/7/o/QtRGma2QlKnKO0+4BfCwr0riiuIB17nXNYSJF2jTcTM\nugPdi8i+vZjjDLgu2grLHwHsuiFl8cDrnMtqOThi2AOvcy675eJcDR54nXNZS0pdU0M28cDrnMti\nvvSPc86lXQ7GXQ+8zrns5jVe55xLp9T2480aHnidc1krlf14s4kHXudcVvOmBuecS7McjLsVN/Du\n1KIhnw1+NNPFyEoH3j8w00XIWt/e0iHTRcha+WXRJJD8fLzlSpGBV9LGyZ4kWi7DOedSKp3z8aZT\ncTXeRYRZe5JRKQVlcc65OMlP+1ieFBd4O6atFM45V4QK1dRgZh+nsyDOOfcPOdqPN+kVKCRtJukS\nSY9I2ixK201Sw7IrnnOuIgttvGW/9E+6JdWrQVIb4HNgJrAd8ARhbfpjgCbAGWVUPudcBVceA2si\nydZ4/w28YGYtgeUx6R8C+6a8VM45F0nVmmvZJNl+vLsB5xeSPhXYMnXFcc65GBW8jXc5/1w7HqA5\nMCt1xXHOuXWUwuXdo2dUwyQtl9QzLq+DpLGSlkj6MlruvSCvqqSXJC2QNEPSVckeW5RkA29/4GZJ\nBTVki5ZEvg/om+Q5nHNug0mJtyRNA+4CXlr//KoL9AFuAeoAw4C3YnbpDjQDGgMHAtdJOjzJYwuV\nbOC9BtgamAFsRFgz/ndgBXBjkudwzrkNlicl3JJhZn3MrB+hY0Cs44ExZtbbzJYRAm0bSS2i/DOB\nO81srpn9AjwPdE3y2EIl1cZrZnMl7QUcDuxCCNg/AgPMbE0y53DOuQ21AWuu1ZU0LOb9c2b2XJKX\naQWMKnhjZoslTQBaSfoL2Co2P3rdKdGxwNiiLpj0JDlRgB0Qbc45lxZJdlqYZWbtSniJGoSusrHm\nAzWjvIL38XmJji3ShgygOFzSJ5KmRNunknxYsXOuTKVhAMUi/tl5oBawMMojLr8gL9GxRUoq8Eq6\nFHgP+Au4J9pmAP0kXZbMOZxzriRS+HCtKGOANuuup+qEgWJjzGwuMD02P3o9JtGxxV0w2RrvDcCV\nZnaGmT0VbWcAV0Z5zjmXciLqUpbgv6TOJeVLqkaYTbGSpGpRT62+QGtJJ0T5twKjzaygjfZVQq+u\n2tFDs/OAnlFeomMLlWzg3QQobNKcTyi8f69zzpWeRKW8xFuSbgaWEiqLp0evbzazmcAJwN3AXGAP\noHPMcbcBE4DJwCDgQTP7CCCJYwuV7MO1/sCxhKHDsY7FH7Y558pQqkaumVl3QnevwvI+AwrtAmZm\ny4Gzo22Dji1KcStQXBTz9n+Eqva+wLdR2p7A/sADG3JB55xLliDpfrrlSXE13lvi3i8jVKP3iEu7\nFLg3xeVyzjkgN+dqKG4i9PrpLIhzzhUmF6eFrLCrDDvnsp/Ehjw8KzeSDrySmhDGJTcCqsTmmdlF\nhRzinHOllnthN/kVKA4hDKAYy7qxydsS+sN9X2alc85VeLnY1JBsP957gPvMbGfC3LwnE2q+gwgB\n2TnnUi70aki8lTfJBt4WwGvR61XARma2mDBK49qyKJhzzpHEPA3lsUacbOBdzLp23emEscgABmyW\n6kI551yBirzm2vfA3sAvwEfAg5JaEobKeRuvc65MFDQ15Jpka7zXAiOj17cBXwPnAH9Hf7piXHbh\nubTcZiv23b3t2rRzu5zKAXvvygF778ourZpywN67rs0b89NoOh7Unva7tWG/PdqybNmyTBS7zNzR\naQcGXr8ffS7Zc23aoa02p++lezLq9g7ssNW6qUxbN6hF74v2oPdFe/DOxXtwUMt6650rT/D2RXvQ\n4/Q25JoLu51Nk623YLedd1ybNmfOHI7ueChtdmjO0R0PZe7cuesdM3zYD9TaKJ++fd5Jd3HLTIVt\najCzX81sePR6oZmdZWbNgePIzd4eKdX5tC682ff99dJeeOU/DPx6OAO/Hs5RxxzHUcccB8CqVau4\n6NwuPPjYkwz9YRT9BnxO5cqVM1HsMvPuiGlc+OqI9dJ++3sRV74xmuGT562XPv7vRXR+5ntOeuo7\nLnhlBLce03K9fp2n79WIiTMXp6Xc6XbaGV3p1//D9dIefvA+DjjoIEb9PI4DDjqIhx+8b23e6tWr\nueWmGzj4kMPSXdQypSS28ibpidCLsAPwWyoKksv2br8vtWvXKTTPzHi37zscd+LJAHz5+afs0HpH\nWu8YanB1NtuMSpUqpa2s6TB88jzmL125XtrEmUuYNGvJP/ZdtnINq9cYAFXz8wiPFYItalVl3+Z1\n+e+wqWVa3kxpv+9+//j/5oP+73Ha6V0AOO30Lrz/3rtr85558gmO7XQ89TbfPK3lLEtS6tZcyyal\nDbyulL75aij1Nt+c7Zo2A2DC+HFI4qROR3BQ+9144pGHMlzCzNtx61r0vXRP+lyyJ3e8N3ZtIL7u\niOY88slvrLEEJ8ghf//9F1vWD6P5t6xfn5kz/wZg2tSpvPdeP87tdkEmi1cmcvHhWlYEXkmTJB0c\nl9ZV0tCY/KWSFkn6S9LLkmpEeQMlnZuJcqdC33fe5PgT103fuXrVar775mueeeFV3v9kEAP692Pw\nwC8yWMLM+9+UBRz3xLd0fvZ7zt2vCVXy89iveV3mLFrBz9OKXWGlwrjumiu58+77cu7XEaRlBYq0\nK09zNRxtZp9JakCYlP1myvnqF6tWreKD9/rx2ZDv1qZt1aABe+2zL5vVrQvAwYd1ZPTIEex3wEGZ\nKmbWmDhzCUtXrKbp5tXZufEmHNiiHvs2r0vV/DyqV83n3hNb8a93il1xpdzbfPMtmDF9OlvWr8+M\n6dOpVy80K4wYPoyuZ5wCwOxZs/j4owHkV8rn6GM7FXe6rCfKZ1NCIsUGXkk7JDh+uwT5KWdmUyV9\nCLRO97VTbdCXn9O0+fZs1WDrtWkHdjiUJx59iCVLllClShW+HjqY8y++PIOlzKwGm1ZjxoLlrF5j\n1N+kGk3qVmfavGU89ukEHvt0AgDtmtSma/tGOR90AY446mhef+0Vrr72Bl5/7RWOPPoYAMaM+33t\nPuefexaHH3FkuQ+6AJTTGm0iiWq8PxH7NOOflCA/5SQ1BI4A+pTg2G5AN4CtGzZKccmK1u2s0/lq\nyCDmzJ7FTts34bobb+X0LmfT9523OP6kk9fbd9Patbnwkis4dP+9kMTBhx7OoYcfkbaypsP9J7Vm\nt21qs+nGlfnsmvY8+cXvzF+6khuP3J7a1avw1BltGTt9ERe8OoKdG2/KOfs1YdVqY40Zd78/lnlL\nVia+SA7oesapDBk8kNmzZtF824bcdEt3rrr2Bs489WReffkltm7YiF5vvJ3pYpa58thdLBGZFR03\nJW2fzEnM7NdSFUKaBNQlDEcuUAX40czax+XPBz4ArjazpZIGAq+Z2Qsbcs22u+xqnw3+LvGOFdCB\n9w/MdBGy1re3dMh0EbLWvnvtxo/Dh6U0Sm7RtLWd/FDiPslPHNdyuJm1K26faIbFp4C9CHPOvANc\nYWarJLUFXgRaEgaKnWNmI6PjBNwHFDxLehG43ooLngkUW+MtbUDdQJ2itYuA8HCNdTf6j3znXMWQ\nwk4LTxEGfdUHNgU+BS6S9AzwLvBotM/5wLuSmpnZCsKv5E6EZdwtOu534JmSFiQrejU451xRUjg7\n2TbA22a2zMxmEKY/aAUcQKiEPmpmy83scUIzasET7S7Av81siplNJSz627VU91Sag7NIvqRqMVtu\nDfVyroIK3cWSGjJcV9KwmK1bIad7DOgsaeOod1RH1gXf0XFNB6OjdFg3B3mBUTF5JZIrgfdpYGnM\n9nJmi+OcS5Uka7yzzKxdzPZcIacaRAiYC4ApwDCgH1CD8Owo1nygYNKQ+Pz5QA2V4qlfVvTjNbMm\nhaT1BHoWlR+z3wFlUyrnXKaJ1Ky5JimP0P//WcJMizWAl4D7CVPd1oo7pBZQMDpnUVx+LWBRaR6u\nbVCNV1INSW38p7xzLl3yktiSUAdoCPSI2nFnE34ZHwGMAXaKq8HuFKUT/Rk7/V2bmLwSSarMkqpL\nepVQRR8e3QCSeki6qTQFcM654qRiyLCZzQImAhdKype0KeGh2ShgILAauExSVUmXRIcVjNV/FbhK\nUgNJWwFXE/0aL6lka7z3Epb/2RuInRz2E+Ck0hTAOeeKoiRmJtuAIcXHA4cDM4HxhHEBV0ZdxjoB\nZwLzgLMJ3VdXRMc9C/QH/kcYVPZBlFZiybbxHgv8n5l9Jym2XeNnwmrDzjlXJlI1cC0aEHFAEXkj\ngF2LyDPgumhLiWQDbz1Cx+N41VNVEOeciycgvxxO+5hIsk0NwwmN0AUKar1nA9+ktETOORejIk8L\neRMwQFKL6JiLJRWM+Ni/jMrmnKvoNmxkWrmR7JprgwkBdnNgKqGRejGwj5n5KsPOuTKjJP4rb5Ie\nQBEtdnlywh2dcy5FcnV596QCr6SNi8s3s3+uUuiccymQipFr2SbZGu8iip/wPPcWenLOZVyFrvES\nZvGJVRnYmTBf7i0pLZFzzhUop70WEkkq8JrZx4Ukvy9pHHA6YUidc86lXC4udlnaaSGHsW6yYOec\nS6mCpoYUTYSeNUo8LaSkKsDpcHQvAAAa3ElEQVTFhO5lzjlXBkSlHKzxJturYSbrP1wTYc2iFYSJ\nJZxzLuVEBW7jBW6Oe7+GMMPP12ZW2BwOzjlXeuW0KSGRhIFXUj6wEhgQLRDnnHNpUyEfrpnZKqAH\nULXsi+Occ+sUNDVU1ElyvicsdzG5DMvinHP/UJFHrvUA/h0tezGcMEHOWmb2c6oL5pxzIneWQo+V\nbOB9O/rzqejPgh4Oil77kGHnXOopLP+Ta5INvC3LtBTOOVeE3Au7CWrxkl6SVNPMfi1uS1dhnXMV\nSxi5lrLFLpHUWdIvkhZLmiBp3yi9g6SxkpZI+lJS45hjqkaxcIGkGZKuKu19JWo+6QJsVNqLOOdc\nSaVqyLCkQ4D7gbOAmsB+wO+S6gJ9CBN+1SFMhfBWzKHdgWZAY+BA4DpJh5fmnhI1NeRiLd85V24o\nlW28twN3mNm30fupAJK6AWPMrHf0vjswS1ILMxtLGJ17lpnNBeZKeh7oCnxU0oIk88CwuHl4nXOu\nzBT0aki0AXUlDYvZuq13HqkS0A6oJ2m8pCmSekjaCGgFjCrY18wWAxOAVpJqA1vF5kevW5XmvpJ5\nuDYj0TeOmXmvBudcmUiyxjvLzNoVk78FYR7xE4F9CaNx3yVMh1CDMAVCrPmE5ogaMe/j80osmcDb\nDZhXmos451xJpaihYWn05xNmNh1A0sOEwDsYqBW3fy1gIWH1nYL3y+LySiyZwNs/VyfC8Qbswv1w\n28GZLkLWan7le5kuQtaa8Wfq62cSKZkW0szmSppC4U2nYwgdCaJrqjqwHaHdd66k6YSRu59Gu7SJ\njimxRG283r7rnMsoSQm3JL0MXCpp86jt9grgfaAv0FrSCZKqAbcCo6MHaxBW2LlZUm1JLYDzgJ6l\nuadEgdcrhc65jFISW5LuBH4AxgG/ACOAu81sJnACcDcwF9gD6Bxz3G2Eh22TgUHAg2ZW4h4NkKCp\nwcxycZi0c64cSVVvMjNbCVwUbfF5nwEtijhuOXB2tKVEiZf+cc65sha6k+XeD28PvM65LLZhQ4LL\nCw+8zrmsloNx1wOvcy57eVODc86lWzld2icRD7zOuazmgdc559JIpGbkWrbxwOucy2ryNl7nnEuv\nHKzweuB1zmU3r/E651wahTXXMl2K1PPA65zLXhu4mGV54YHXOZfVci/seuB1zmWxguXdc40HXudc\nVsu9sOuB1zmX7XIw8nrgdc5lNW9qcM65NMu9sJt4zTXnnMusFC66BiCpmaRlkl6LSTtV0mRJiyX1\nk1QnJq+OpL5R3mRJp5b2ljzwOueyVoirif/bQE8SFr0M15BaAc8CZwBbAEuAp+L2XxHlnQY8HR1T\nYt7U4JzLXimej1dSZ2Ae8DXQNEo+DehvZoOjfW4BfpFUE1hDWIG4tZktAoZKeo8QpG8oaTm8xuuc\ny2pS4g2oK2lYzNbtn+dRLeAO4Oq4rFbAqII3ZjaBUMNtHm2rzWxczP6jomNKzGu8zrkslnRTwiwz\na5dgnzuBF83sT61fja4BzI/bdz5QE1hdTF6JeeB1zmW1VDQ1SGoLHAzsXEj2IqBWXFotYCGhqaGo\nvBLzwJsGl114Lp98NIC69TZn6Pcj16Y//0wPXnj2afLz8znksI50v+s+Vq5cyRUXd2P0qBGsWrWa\nk085nSuuuT6DpU+vxx99hJ4vv4AkWrXekedeeJlq1aoBcOXll9LrlZeZNW9RhktZdh48tS0dWm/B\n7IXLOeTegQDceOwOHLzjFqxcZUyetZhrXh/BgqWraNN4U+7r3AYIwemRAb/y8egZ1N+0Go+csQv1\nalXFzPjPV5N5adDEDN5VyZWg00JRDgCaAH9Etd0aQCVJOwAfAW3WXlPaFqgKjCME3nxJzczst2iX\nNsCY0hTG23jToPNpXXir7/vrpQ0ZPJAPP+jP4G9/5KsfRnHx5VcB8G7fd1i+YgVDvhvJ50O+45WX\nn+ePyZMyUOr0mzp1Kk89+ThffTuM4SN/YvXq1fR+600Ahg8bxvx58zJcwrLX+7s/OPOpb9dLG/Lr\nTA65ZyCH3TeQiX8v4uJDmgHw67SFHPXgYDreP4gzn/qWezu3oVKeWL3GuKvvGDrc/SXH/nsIZ+63\nDc22rJGBu0mR1HQnew7YDmgbbc8AHwCHAa8DR0vaV1J1QjtwHzNbaGaLgT7AHZKqS9oHOBboVZpb\n8sCbBnu335fateusl9bzhWe5/KrrqFq1KgD16m0OgCSWLF7MqlWrWLZ0KZUrV6FmzfhfOrlr1apV\nLF26NPy5ZAn1t9qK1atXc+MN13L3fQ9kunhl7vsJc5i3ZMV6aUPGzmT1GgPgx0lz2XLTjQBYtnL1\n2vSqlSth4SV/L1jOT1NCs+Ti5asZP2MhW26yUZruIPXyoqkhi9sSMbMlZjajYCM0Lywzs5lmNga4\ngBCA/ya0314Uc/hFwEZR3hvAhdExJb+n0hzsSm7C+HF88/VQDj1wb44+/CB+HB66FR7T6QQ2rl6d\nVk0b0naHbbn4siupXadOgrPlhgYNGnDFldfQfNtGbNOwPrVqbcLBhxzK00/24MijjqF+/fqZLmLG\nnbxnIwb+/Pfa920bb8pnNx7AJ/86gBvfGrU2EBfYus5GtNp6E0ZMnpvuoqZMisdPAGBm3c3s9Jj3\n/zGzRmZW3cyONbM5MXlzzKxTlNfIzP5TqhsijYFX0iRJKyTVjUsfKckk7SVpgaRKMXnPF5H2TPS6\nlaRPJM2VNE/ScElHpOueSmPVqtXMnzeXj7/4itvvuo9zu5yKmfHjsO+pVCmPn377g+E//cZTTzzK\npIm/Z7q4aTF37lze7/8uv/w2kd//mMbiJYt5vder9Plvby665NJMFy/jLjm0GavWGH2HTVmbNnLy\nPA6+ZyBHPziYiw9tRtX8df+kN65SiWfP2Y3b+4xh0bJV6S9wKiQTdcvhmOJ013gnAqcUvJG0I6EK\nDzALqATsErP/vsC0uLT9gMHR6/7Ap4QRJZsDlwELyqLgqbZVgwYcecxxSGKXdruTl5fH7Fmz+G/v\nN+lw8GFUrlyZevU2Z48992LkiOGZLm5afPH5ZzRpsg316tWjcuXKdOp0PHfecRu/TxhPqxZN2b5p\nE5YsWUKrFk0TnyzHnLh7Qzq03oLLXvmx0Pzxfy1iyfLVbF8/9HLKzxPPnrsbfYdN4aNR09NZ1JQr\ng5FrGZfuwNsLODPmfRfg1ej1SuBbQmBF0uZAFeCtuLTmwOCo5rwN8LyZrYi2r8xsaFrupJQ6HnUM\nQwZ9CcD438axYsUKNqtbl623bsSQQV9iZixevJhhP3xPs+bbZ7i06dGwYSO+//5blixZgpnx5Ref\nc9nlVzFpygx+HT+JX8dPYuONN2bM2PGZLmpa7d+yHhce3JRznvueZStXr01vuNnGVIoWJGtQeyO2\n26IGf85ZCsCDp7Vl/IyFvPBl+f61JJIeQFGupLs72bfAGZJaErpqnAy0B+6K8gcTguy/oz+HRtsl\nMWkTzWyKQp+Q8cBrkl4AvjGzv4q7eDSapRvA1g0bpfjWinbeWafz1ZBBzJk9ix23b8L1N97KaWec\nxWUXnUv73dtSuUplejz7EpI4u9uFXHZhSDczTjm9C61a75S2smbS7nvswXHHn8heu+9Cfn4+bdrs\nzDnn/WMAUk57ousu7NW0LrVrVOG7Ow7h4QG/cvGhzaiSn8frF+8FwIhJc7nxrdHstm0dLjqkKStX\nG2vMuOnt0cxdvILdtq3DCbs35JepC/jw+v0BeKD/L3wZ0zZcnpTHwJqIzCzxXqm4kDQJOBfYE6gO\nDCIM3etIqO1uQ+hn91+gLvAo8BPhKeIEYMsorZaZnRWdc2vCeOmO0fFDgXNi+tsVqe0uu9rng79L\n2f3lkurVvHt3UZpf+V6mi5C1Zrx1FSv+Hp/SMNm6zS72zkeJf8S23Kr68CRGrmWNTPRq6AWcCnRl\nXTNDgW8JHZtbE2q3Q6KJKf6MSSto38XMppjZJWa2HdAYWFzIOZ1z5VguNjWkPfCa2WTCQ7YjCB2T\nY/OWEaZrOwqob2Zjo6whUdpOxATeuGP/JEzf1rpsSu6cy4Qc7NSQsX685wAHRaNC4g0GriBM21Zg\naJQ2I5o5CEm1Jd0uqamkvOhh29mEWrNzLlfkYOTNSOA1swlmNqyI7EGErmGxDTtDo7TY2u4KQpvw\nZ4QuZD8BywlNGM65HCClZuRatknbUxQza1JE+ipivrPM7GPivsOi3grxaYsJ3dGcczms/IXVxPzx\ntXMuu+Vg5PXA65zLYuVzZFoiHnidc1mtHDbhJuSB1zmXtQqGDOcaD7zOuazmTQ3OOZdmXuN1zrk0\ny8G464HXOZfFFJbDyjUeeJ1zWStXH675mmvOuayWiqkaJFWV9KKkyZIWShohqWNMfgdJYyUtkfSl\npMZxx74ULUM2Q9JVpb0nD7zOuayWomkh8wnTy+4PbALcArwtqUk0wVafKK0OMIyw8k2B7kAzwtSz\nBwLXSTq8NPfkTQ3OuayWiu5k0dwu3WOS3pc0EdgV2AwYY2a9ASR1B2ZJahFNTXsmcJaZzQXmSnqe\nMBnXRyUtj9d4nXNZLckab11Jw2K2YteMkrQFYf3GMUArYFRBXhSkJwCtJNUGtorNj163Ks09eY3X\nOZe1NqApYVayS/9Iqgy8DrxiZmMl1QBmxu02H6hJWBGn4H18Xol54HXOZbVUjlyTlEdYfmwFYRFd\ngEVArbhdawELo7yC98vi8krMmxqcc9ktRStQRCuTvwhsAZxgZiujrDFAm5j9qgPbEdp95wLTY/Oj\n12NKfD944HXOZbkUrvzzNNASONrMlsak9wVaSzpBUjXgVmB0zJqPrwI3R8uNtQDOA3qW4pY88Drn\nslniZX+SWfon6pd7PtAWmCFpUbSdZmYzgROAu4G5wB5A55jDbyM8bJtMWJrsQTMrcY8G8DZe51wW\nS9XItWh18yLPZGafAS2KyFtOWEj37NKXJPAar3POpZnXeJ1zWS0X52rwwOucy2o+EbpzzqWRBHm5\nF3c98DrnspwHXuecSy9vanDOuTTzh2vOOZdmORh3PfA657Kbr7nmnHNplKtrrsnMMl2GjJA0kzD2\nOhvUBWZluhBZyj+b4mXT59PYzOql8oSSPiLcYyKzzKxUy/GkU4UNvNlE0rBkJ3GuaPyzKZ5/PuWT\nz9XgnHNp5oHXOefSzANvdngu0wXIYv7ZFM8/n3LI23idcy7NvMbrnHNp5oHXOefSzAOvc86lmQfe\nLKRcHCPpnFvLA28WMn/i6UpI0tHREuQui/lcDVlE0nnAlsB3wDAzm5PhIrlyRNJbQD3gbkkTo9Vx\nXRby7mRZQlI/oAGwCNgceMzMnpMkrwG7RKKguxVwJLDUzFbG5eeZ2ZqMFM79gzc1ZAFJ/YGNzGw3\nMzsQGAScLym/ogddSSdKapXpcmQzSe2ARsDhZrbAzFZKqidpf0mHAJjZGkn+7z1LeFNDhkk6kFBL\nOSAm+V1C7bc6MD8DxcoKkloDdwLvSXrRzMZlukxZyoCaZrZY0ubALsCrwFygtqTBZnai13izh38D\nZpiZfQncBnwsqa2kysDzwBdmVmGDLoCZ/QScCewJnCOpeUGepEoxrzfJQPEyTtKVks42s+FANUmj\ngZeBR4AngP2BU4HdC2q+Ljt4jTeDCtrdzOzO6GfgMGApcKOZPRHtU+HaeCXdA2wHzAQeBS4nBBKT\n9LKZ/Wpmq6N9LwfaSLrAzFZkrNBpJqk6sAeh8vQSoZZ7FzAKmGhmX0T7LSHMO70wQ0V1hfCHaxkg\naVdgnJktjEu/FHgMaG1mP0uqHP+QJNdJepcw8fXnQFtCYOlEqL3dBnwDvGRmv0q6BngA2NXMRmSo\nyBkjaTtgOPCgmd1dxD7nA1cAB5vZ1HSWzxXNa7xpJuku4EZgqKS7gdFmNh3AzJ6QVA8YKekwM/uy\nIj2NlvQO0KBgYm9JnYD7gPpm9rkkA7oD86K2zAuA3SpS0JV0AfACkGdmEyRdCHSX9E1BLTfarz2w\nD3AtcKgH3ezigTf9fgImAnOAnsAISV8APYDlZnarpNXA55L2M7OhmStq+kiqAxwGPCZpYzNbAvwM\nLAFqRE0uX0SD+h4nPHxsH7VvVgiSjgeeIrR5fyZpMNCb8Ll1lDTczOZHzRAdo/0ONLP/ZazQrlDe\n1JAmkqqZ2TJJ2wJfAtcAPwKnA10J7ZkfEn42LpJ0CfC5mf2SqTKni6RbCU0IU4D3CF9IHwL9gB5m\n9kDc/u0Ia2xNSm9JM0tSQ8JnspgQcDsSarRbE764u5rZV9G+GxG6KPognCzkvRrSQNLzwGkAZvY7\n8BChbXKNmd1OCMK7AicC0yVdBTxdQYJubWAj4G6gDnAscDbwNfB6QdCN7YNqZsMqWtAFMLM/gUuB\nSsDvwNtAf6AKMA14UlLNaN+lHnSzlwfeMhY9LNrZzF6MSf4S+AtoJGk/Qvefi4AdCe2//Qqe2uc6\nM5sLPEnou/wwUBs4FJhKaMutE+1XIdq540m6RdI3kppGTTBfA+8DO5hZT+AS4GDCSsM7Ef4/clnO\nmxrKkKS+QD0zax+TVsXMVkh6CjiDMET4JjN7KVPlzIRoVN6qmPeVgXuAo4CzgAWE2twbhOHTMzNS\n0AySdD1hFOPdQGXgM+Bpwnwe9wFXmtk4SY2AZoTuZGeZ2dgMFdklyQNvGZH0AuGhR5OYPqfbEh4M\nXQIsBz4FBpvZRRWpv66kewkPGd8BVpiZSdqK0Ob9E1ATuJpQi/uGUBO+uyLVeqM2/seBT4BuwNFA\nB6Al0JkwsGRvYP+C/ssVqQdMeedNDWWnF+GJ/KEAkrYGBgMjo/bJecAIQu2lok0F+SVwK3B0FHQb\nEmZke4zwZdWP0AZenzBI4I0KGFA+AIYAuxOaFj4gdJ/7gBCM5xCaZS4omL+5An5G5ZYH3jJiZoMI\nNZWHo76X3wBPmdnNUe12KSHYHChp04o0+bmZfQKcB9wZfTZfA8+Y2b3Rr4P/AF8ANwN/mtn4zJU2\n/aLh0NMIQbYL4Qt6MFDdzK4h/GJqCTQHjgeqZaioroS8qaGMRQ/PBgDvm1nnmPQrCT8XO5vZr5kq\nXyYV9tkU/FyOmh6Wm9nsjBYyjSTtBXwf0zR1FnAlodb7LOEhWnszmyipAaGf7k8V9f+f8swDbxpI\n2ocw2ugmM+sT1fLuJQzjrDADAAoTfTbPAzcBH0cDJyocST0JX8QvEHq83BZ9Ad0HTDezxxTmbN6J\nMBKtQv0KyDUeeNNE0v6EiV5GEqaBPMTMfsxsqbJD9Nk8RXgq/9+KNNlNAUl7E9puXwW2AQR8TGjn\nrmJmV0T7fUp4LrBzbK8QV7544E2jaO7d/xAmrB6V6fJkE0kHE34FHBQ/eVBFEdX+nwGuIkxsXpPQ\nzl0HOMzMPo3229rMpmSsoK7UPPCmWcHQ4UyXIxvFzNFQYUVfzo8Dt5pZX0ltgaaEIdRLvedCbvDA\n61yWkXQAod37X2b2ToaL48qAz07mXJYxs4GSzgWejkb0Vch271zmNV7nspS3e+cuD7zOZTFv985N\nHnidcy7NfMiwc86lmQde55xLMw+8zjmXZh54nXMuzTzwOudcmnngdSUm6SdJ3WPeT5J0TQbK0U6S\nSWpShtfoLumnFJwnI5+Ryy4eeHOIpJ5RADJJKyX9LukhSdXTVITdCLOMJSSpq6RFZVye2OsNlNQj\nXddzrjg+ZDj3fEZYRLMysC9hftfqwIWF7SypspmtTMWFK+KClM6VhNd4c89yM5thZn+a2X+A14FO\nECZfiWrDR0j6XtIKwhpnSDpa0nBJyyRNlHS3pCoFJ5W0uaR3JS2VNFnS2fEXjv8ZLamWpKclTY/O\n+4ukk6NJYF4GqsfU0LtHx1SRdL+kKZIWS/pB0mFx1zlc0tjonEMIS+CUiqT7JP0a3d8kSQ9I+seS\nOpLOlfRHtF8/SXXj8s+S9HNUtnGSrpTk/87cerzGm/uWEmq/se4nrOI7HlgYBbbXgcsJa3s1IswL\nWxUoCKQ9gcaE5WeWAI8ATYq6aLSG3IeEBRnPAsYB2xPWB/sauIKwnPt20SEFzQ4vR2mnAlOAI4D+\nknYzs1HRwpj9CLN3PUlYkeHh5D+OIi0GzgamAjsQ7n85cEvMPk2A04FjgY2B54CXgGOiez4PuAO4\nFBgOtI7KuRLwZg63jpn5liMbITi+H/N+d8IS6W9F7w8ADDgh7rjBwC1xaZ0IwVCEGqUB+8TkNwZW\nA91j0iYB10SvDwHWAC2LKGtXYFFc2nbRMY3i0vsRFgqFEKzHEQ13j9JujsrXpJjPZiDQYwM+ywuA\n8THvu0f32ygmrX103WbR+z+AM+LOcwXwc2GfkW8Vd/Mab+45PHpolU+o6b5LqIHFGhb3fldgd0nX\nx6TlARsRlplpSQiI3xdkmtlkSdOKKcfOhLXCftmAsu9CCPQ/xy26XJWw6jBRWb41s9hJRr7ZgGsU\nStKJhCDZFKgBVIq2WFPN7I+Y998RfblImgc0BJ6V9HTMPvmEe3JuLQ+8uWcwYVn5lcA0K/zB2eK4\n93nA7UDvQvadSckCR0mOySPUIHcjlD/W0lKct1iS9gTeJHwGVwLzCM0HD23AaQracQuWq3euSB54\nc88S2/AVaH8EWhR1nKRfCIFlN6KgIqkRsFWCc9aX1LKIWu8K/lmjHEEIrFua2ZdFnPdn4ARJiqn1\n7llMOZKxD6E2e2dBgqTGhezXQFJDM/szer874XP5xcz+kjQV2M7MXi1leVyO88DrIDwQel/SZOBt\nYBXhwdDuZnadmf0q6SPCz+huhNrnw6yrhRbmc8JP8f9KupLQLtsUqG5m/QhtndUkHUIIuEvMbJyk\n14Gekq4mBO86hLbp382sD+Gh19XAo5KeAnYk1DKTUTdawyzW31HZGkg6jdBscRhwSiHHLwVekXQV\noRnmGeADM/styu8OPBE1OwwgNPXsAjQws3uTLKOrALybi8PMPiYsOX8goR33e+AGwsOiAl2BiYS2\n1v6E1ZInFXPONUBH4CvgNeAX4DGgSpT/NSFwvUFozrguOvQsQs+GB4CxwPvAfsDk6Lg/gOOBw4FR\nhKaBG5K81ZMJQT52u8rM+gMPAo8CowkPBm8t5PhJhCaJ/tHn8HtU3oJ7foHQM+KMqGxDCM0+E5Ms\nn6sgfCJ055xLM6/xOudcmnngdc65NPPA65xzaeaB1znn0swDr3POpZkHXuecSzMPvM45l2YeeJ1z\nLs3+H9nfTJhALZAFAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2a387d6a128>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Confusion Matrix\n",
    "conf_mx = confusion_matrix(y_test, y_pred)\n",
    "plot_conf_matrix(conf_mx, train.author.unique(), title=\"MNB_GS Confusion Matrix\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}