diff --git a/documentation/tutorials/kaggle_beginner_example_classification.ipynb b/documentation/tutorials/kaggle_beginner_example_classification.ipynb new file mode 100644 index 00000000..bd796f04 --- /dev/null +++ b/documentation/tutorials/kaggle_beginner_example_classification.ipynb @@ -0,0 +1,1804 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##### Copyright 2022 The TensorFlow Authors." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", + "# you may not use this file except in compliance with the License.\n", + "# You may obtain a copy of the License at\n", + "#\n", + "# https://www.apache.org/licenses/LICENSE-2.0\n", + "#\n", + "# Unless required by applicable law or agreed to in writing, software\n", + "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", + "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", + "# See the License for the specific language governing permissions and\n", + "# limitations under the License." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "MDBzBKC_pnXl" + }, + "source": [ + "# Structured Data Classification using TFDF\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " View on TensorFlow.org\n", + " \n", + " Run in Google Colab\n", + " \n", + " View on GitHub\n", + " \n", + " Download notebook\n", + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "MA9_xqWRpqZU" + }, + "source": [ + "## Introduction\n", + "\n", + "[TensorFlow Decision Forests](https://www.tensorflow.org/decision_forests)\n", + "is a collection of state-of-the-art algorithms of Decision Forest models\n", + "that are compatible with [Keras APIs](https://www.tensorflow.org/api_docs/python/tf/keras)\n", + ".\n", + "The models include [Random Forests](https://www.tensorflow.org/decision_forests/api_docs/python/tfdf/keras/RandomForestModel),\n", + "[Gradient Boosted Trees](https://www.tensorflow.org/decision_forests/api_docs/python/tfdf/keras/GradientBoostedTreesModel),\n", + "and [CART](https://www.tensorflow.org/decision_forests/api_docs/python/tfdf/keras/CartModel),\n", + "and can be used for regression, classification, and ranking tasks.\n", + "For an introduction to [TFDF](https://www.tensorflow.org/decision_forests) without Kaggle, please refer to this [tutorial](https://www.tensorflow.org/decision_forests/tutorials/beginner_colab).\n", + "Decision Forests are a family of tree-based models including Random Forests and Gradient Boosted Trees. They are the best place to start when working with tabular data, and will often outperform neural networks.\n", + "\n", + "In this example we will use TensorFlow to train each of these on a dataset you load from a CSV file. This is a common pattern in practice. Roughly, your code will look as follows:\n", + "\n", + "```\n", + "import tensorflow_decision_forests as tfdf\n", + "import pandas as pd\n", + " \n", + "dataset = pd.read_csv(\"project/dataset.csv\")\n", + "tf_dataset = tfdf.keras.pd_dataframe_to_tf_dataset(dataset, label=\"my_label\", task=tfdf.keras.Task.CLASSIFICATION)\n", + "\n", + "model = tfdf.keras.RandomForestModel()\n", + "model.fit(tf_dataset)\n", + " \n", + "print(model.summary())\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "mrTx_bPrtd17" + }, + "source": [ + "### Setup" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "dl6_Mdy7sUC7" + }, + "source": [ + "#### Install TensorFlow Decision Forests" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "lSDtXxIDseKq" + }, + "outputs": [], + "source": [ + "!pip install tensorflow_decision_forests --quiet" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Zr0eiHcyvG1m" + }, + "source": [ + "#### Import the library" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "IA1LNshiumEA" + }, + "outputs": [], + "source": [ + "# Scientific computing # \n", + "import numpy as np # Numpy Documentation - https://numpy.org/doc/stable/ \n", + "\n", + "# - Data processing - #\n", + "import pandas as pd # Pandas Documentation - https://pandas.pydata.org/docs/\n", + "\n", + "# ---- Tensorflow ---- #\n", + "import tensorflow as tf\n", + "import tensorflow_decision_forests as tfdf" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "CjdtV-KWvcWA" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "TensorFlow v2.9.1\n", + "TensorFlow Decision Forests v0.2.7\n" + ] + } + ], + "source": [ + "print(\"TensorFlow v\" + tf.__version__)\n", + "print(\"TensorFlow Decision Forests v\" + tfdf.__version__)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "vzX1j4YewLSr" + }, + "source": [ + "### Download the Titanic dataset\n", + "The [Titanic dataset](https://www.kaggle.com/competitions/titanic/overview/description) is an example of a binary classification problem in supervised learning. We are classifying the outcome of the passengers as either one of two classes, survived or did not survive the Titanic." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "_vmqf8o60D37" + }, + "source": [ + "To run this notebook, you need to have a Kaggle account.\n", + "\n", + "If you do not have an account, you can create one here: [Kaggle Register](https://www.kaggle.com/account/login?phase=startRegisterTab&returnUrl=%2F) \n", + "\n", + "In order to get a token to use in the following cell, check out the [Authentication Section](https://www.kaggle.com/docs/api#authentication) of Kaggle API documentation" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "cellView": "form", + "id": "ekAcuqTFvt3p" + }, + "outputs": [], + "source": [ + "#@title Enter your Kaggle token in order to fetch the dataset\n", + "\n", + "username = '' #@param {type:\"string\"}\n", + "key = '' #@param {type: \"string\"}" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "cellView": "form", + "id": "JKyVN-lC0HOC" + }, + "outputs": [], + "source": [ + "#@title Configure Kaggle\n", + "try:\n", + " from google.colab import files, drive\n", + "\n", + " # Install and Configure Kaggle\n", + " import json\n", + "\n", + " token = {\n", + " \"username\":username,\n", + " \"key\":key\n", + " }\n", + "\n", + " # Installing kaggle\n", + " !pip install kaggle &> /dev/null\n", + "\n", + " # Creating .kaggle if necessary\n", + " !if [ -d .kaggle ]; then echo \".kaggle exists\"; else echo \".kaggle does not exist ... Creating it\"; mkdir .kaggle; if [ -d .kaggle ]; then echo \"Successfully created\"; else echo \"Error creating .kaggle\"; fi; fi\n", + "\n", + " with open('/content/.kaggle/kaggle.json', 'w') as file:\n", + " json.dump(token, file)\n", + "\n", + " # Creating .kaggle if necessary\n", + " !if [ -d ~/.kaggle ]; then echo \" ~/.kaggle exists\"; else echo \" ~/.kaggle does not exist ... Creating it\"; mkdir ~/.kaggle; if [ -d ~/.kaggle ]; then echo \"Successfully created\"; else echo \"Error creating ~/.kaggle\"; fi; fi\n", + " !cp /content/.kaggle/kaggle.json ~/.kaggle/kaggle.json\n", + "\n", + " # kaggle configuration\n", + " !kaggle config set -n path -v{/content}\n", + "\n", + " # Changing mode\n", + " !chmod 600 /root/.kaggle/kaggle.json\n", + "except Exception:\n", + " pass" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "cellView": "form", + "id": "J501lMS40lUR" + }, + "outputs": [], + "source": [ + "#@title Download Dataset\n", + "import os\n", + "\n", + "DOWNLOAD_LOCATION = \"/root/Downloads/\"\n", + "\n", + "if os.path.exists(DOWNLOAD_LOCATION):\n", + " if os.path.isdir(DOWNLOAD_LOCATION):\n", + " print(\"{} exists and is a directory\".format(DOWNLOAD_LOCATION))\n", + " else:\n", + " print(\"{} exists but is not a directory!!!\".format(DOWNLOAD_LOCATION))\n", + "else:\n", + " print(\"{} does not exist ... Creating it\".format(DOWNLOAD_LOCATION))\n", + " os.makedirs(DOWNLOAD_LOCATION)\n", + "\n", + "# Downloading\n", + "!kaggle competitions download -c titanic -p {DOWNLOAD_LOCATION}\n", + "\n", + "# Extracting archives\n", + "!cd {DOWNLOAD_LOCATION}; unzip -qq \\*.zip; rm -f *.zip" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "PCFSJUjl2fuT" + }, + "source": [ + "## Load the dataset\n", + "Note: Pandas is practical as you don't have to type in name of the input features to load them. For larger datasets (>1M examples), using the [TensorFlow Dataset](https://www.tensorflow.org/api_docs/python/tf/data/Dataset) to read the files may be better suited." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "18QhsN2L16wH" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Full train dataset shape is (891, 12)\n" + ] + } + ], + "source": [ + "train_file_path = os.path.join(DOWNLOAD_LOCATION, \"train.csv\")\n", + "train_full_data = pd.read_csv(train_file_path)\n", + "print(\"Full train dataset shape is {}\".format(train_full_data.shape))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "J8KqgoL95mhw" + }, + "source": [ + "The data is composed of 12 columns and 891 entries. We can see all 12 dimensions of our dataset by printing out the first 3 entries using the following code: \n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "v4rywCtW2pfK" + }, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "
\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
PassengerIdSurvivedPclassNameSexAgeSibSpParchTicketFareCabinEmbarked
0103Braund, Mr. Owen Harrismale22.010A/5 211717.2500NaNS
1211Cumings, Mrs. John Bradley (Florence Briggs Th...female38.010PC 1759971.2833C85C
2313Heikkinen, Miss. Lainafemale26.000STON/O2. 31012827.9250NaNS
\n", + "
\n", + " \n", + " \n", + " \n", + "\n", + " \n", + "
\n", + "
\n", + " " + ], + "text/plain": [ + " PassengerId Survived Pclass \\\n", + "0 1 0 3 \n", + "1 2 1 1 \n", + "2 3 1 3 \n", + "\n", + " Name Sex Age SibSp \\\n", + "0 Braund, Mr. Owen Harris male 22.0 1 \n", + "1 Cumings, Mrs. John Bradley (Florence Briggs Th... female 38.0 1 \n", + "2 Heikkinen, Miss. Laina female 26.0 0 \n", + "\n", + " Parch Ticket Fare Cabin Embarked \n", + "0 0 A/5 21171 7.2500 NaN S \n", + "1 0 PC 17599 71.2833 C85 C \n", + "2 0 STON/O2. 3101282 7.9250 NaN S " + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "train_full_data.head(3)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "IYqyU_6MOrH8" + }, + "source": [ + "* 8 feature columns named `Pclass, Sex, Age, SibSp, Parch, Fare, Cabin, Embarked`.\n", + "* Label column named `Survived`.\n", + "* We will drop the following unnecessary columns : `PassengerId`, `Name` and `Ticket`" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "shj-eSteOqPE" + }, + "outputs": [], + "source": [ + "train_full_data = train_full_data.drop(['PassengerId', 'Name', 'Ticket'], axis=1)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "SYvo-ty6QiHN" + }, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "
\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
SurvivedPclassSexAgeSibSpParchFareCabinEmbarked
003male22.0107.2500NaNS
111female38.01071.2833C85C
213female26.0007.9250NaNS
\n", + "
\n", + " \n", + " \n", + " \n", + "\n", + " \n", + "
\n", + "
\n", + " " + ], + "text/plain": [ + " Survived Pclass Sex Age SibSp Parch Fare Cabin Embarked\n", + "0 0 3 male 22.0 1 0 7.2500 NaN S\n", + "1 1 1 female 38.0 1 0 71.2833 C85 C\n", + "2 1 3 female 26.0 0 0 7.9250 NaN S" + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "train_full_data.head(3)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Qs070SbkMJix" + }, + "source": [ + "Refer to [Kaggle](https://www.kaggle.com/competitions/titanic/data) for a comprehensive guide to the data." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "cwdbYZeTJP89" + }, + "source": [ + "## Exploratory Data Analysis (EDA)\n", + "Data scientists use exploratory analysis techniques to analyze and visualize large datasets. This process helps them identify the main characteristics of their data sets and develop effective strategies to get the answers they need. It can also help them spot anomalies and test hypotheses.\n", + "\n", + "For this dataset, there are some amazing notebooks already available on Kaggle. One of them is [EDA is fun](https://www.kaggle.com/code/prashant111/eda-is-fun#EDA-is-fun) by Prashant Banerjee." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "2EpAa_q55Ke8" + }, + "source": [ + "## Prepare the dataset\n", + "This dataset contains a mix of numeric, categorical and missing features. TF-DF supports all these feature types natively, and no preprocessing is required. This is one advantage of tree-based models; making them a great entry point to TensorFlow and ML." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "uQMX8Md3ISq0" + }, + "source": [ + "Convert the values stored in the `Survived` column to a list of values, where the list does not allow for duplicates. `Survived` has one of two values, 0 or 1." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "YmrDp4SL7hTw" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Label classes: [0, 1]\n" + ] + } + ], + "source": [ + "label=\"Survived\"\n", + "classes = train_full_data[label].unique().tolist()\n", + "print(f\"Label classes: {classes}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "0NGJhK0R58Oa" + }, + "source": [ + "Split the dataset into training and testing:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "CW3ofmmI5xIr" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "611 examples in training, 280 examples in validation.\n" + ] + } + ], + "source": [ + "def split_dataset(dataset, test_ratio=0.30):\n", + " test_indices = np.random.rand(len(dataset)) < test_ratio\n", + " return dataset[~test_indices], dataset[test_indices]\n", + "\n", + "train_ds_pd, val_ds_pd = split_dataset(train_full_data)\n", + "print(\"{} examples in training, {} examples in validation.\".format(\n", + " len(train_ds_pd), len(val_ds_pd)))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "I0ZrYmer6tMp" + }, + "source": [ + "There's one more step required before you can train your model. You need to convert from Pandas format (`pd.DataFrame`) into TensorFlow format (`tf.data.Dataset`). A single line helper function that will do this for you: \n", + "\n", + "```\n", + "tfdf.keras.pd_dataframe_to_tf_dataset(your_df, label='your_label', task=tfdf.keras.Task.CLASSIFICATION)\n", + "```\n", + "\n", + "This is a high [performance](https://www.tensorflow.org/guide/data_performance) data loading library which is helpful when training neural networks with accelerators like [GPUs](https://cloud.google.com/gpu) and [TPUs](https://cloud.google.com/tpu). It is not necessary for tree-based models until you begin to do distributed training.\n", + "\n", + "Note that tf.data is a bit tricky to use, and has a learning curve. There are guides on [tensorflow.org/guide](https://www.tensorflow.org/guide) to help." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "DyAHpZ0R6B5R" + }, + "outputs": [], + "source": [ + "train_ds = tfdf.keras.pd_dataframe_to_tf_dataset(\n", + " train_ds_pd, \n", + " label = label, \n", + " task = tfdf.keras.Task.CLASSIFICATION)\n", + "\n", + "val_ds = tfdf.keras.pd_dataframe_to_tf_dataset(\n", + " val_ds_pd, \n", + " label = label, \n", + " task = tfdf.keras.Task.CLASSIFICATION)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "3m46QYDz8IB4" + }, + "source": [ + "## Create and train a Random Forest model " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "11yxinBK78qU" + }, + "outputs": [], + "source": [ + "model = tfdf.keras.RandomForestModel(task = tfdf.keras.Task.CLASSIFICATION)\n", + "model.compile(metrics=[\"accuracy\"]) # Optional, you can use this to include a list of eval metrics" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "_tQfGooA8OI2" + }, + "outputs": [], + "source": [ + "model.fit(x=train_ds)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "YoqPROtT9A33" + }, + "source": [ + "## Visualize your model\n", + "One benefit of tree-based models is that you can easily visualize them. The default number of trees used in the Random Forest is 300. You can select a tree to display below." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "Cwv7-NXc8WUq" + }, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "
\n", + "\n" + ], + "text/plain": [ + "" + ] + }, + "execution_count": 16, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "tfdf.model_plotter.plot_model_in_colab(model, tree_idx=0, max_depth=3)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "RtGEzEGU9FsI" + }, + "source": [ + "## Evaluate the model on OOB data and the validation dataset\n", + "\n", + "Let's plot accuracy on OOB evaluation dataset as a function of the number of trees in the forest. One of the nice features about this particular hyperparameter is that larger values are usually better, and come with little risk aside from slowing down training." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "4nOZy6lX9CwJ" + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEGCAYAAAB/+QKOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXiU9bn/8fdNIAk7QgCRRXYBZVOKO24VcaUKKtjN1rpVcatWe05rrT2etnoUFXf9tXZTRNxQUamC4oIKyg4SAiKyJUHEJEACSe7fH88THMMkmYRMJsl8XteVi3m2mfvJhLnnu5u7IyIiUl6TRAcgIiL1kxKEiIhEpQQhIiJRKUGIiEhUShAiIhJV00QHUFsyMjK8Z8+eiQ5DRKRB+eSTT7a6e8doxxpNgujZsycLFixIdBgiIg2KmX1R0TFVMYmISFRKECIiEpUShIiIRKUEISIiUSlBiIhIVEoQIiISVVwThJmNMbNVZpZlZrdEOd7DzOaY2UIzW2JmZ4T7O4T7C8zsgXjGKCIi0cUtQZhZCvAgcDowCJhoZoPKnfZbYJq7DwcmAA+F+wuB3wE3xis+kfpmd3Epz32ygYKi4kSHUqFVW/J5+uP17CkpTXQoUgfiWYIYCWS5+1p33w1MBcaWO8eBNuHjtsAmAHff4e7vESQKkaTw1Edf8KtnF3Pzc0uob+u0FJeU8tDbWZw15V1+8/xSzn3ofTKz8xMdlsRZPBNEV+DLiO0N4b5ItwE/MrMNwExgUnVewMwuM7MFZrYgNzd3f2IVSaii4hIeeWctrdOa8uqSzTwz/8uqL6oja3MLOP/Redz5+ipOHdSZey4YyubthZw15T0em7uGktL6lcyk9iS6kXoi8KS7dwPOAP5pZjHH5O6PufsIdx/RsWPUqUREGoRnF2xgS14hD/zwcI7rm8FtLy9P+Df00lLnb+9/zhn3v8va3B3cN2EYD150OOcd3o03rh/Fif078r8zP+PCR+exbuuOhMYq8RHPBLER6B6x3S3cF+kSYBqAu88D0oGMOMYkUu/sKSnl4bfXMKx7O0b1y+CeC4fSKq0pVz/1KYV7ShIS04avd/LDJz7iDy+v4KjeHZh1/SjGDuuKmQGQ0SqNR398BJMvHMqq7HxOv+9d/vnhF/Wuakz2TzwTxHygn5n1MrNUgkboGeXOWQ+cAmBmAwkShOqKJKm88OlGNm7fxbWn9MPM6NQ6nbsvGEZmdgG3v7KiTmNxd56Zv54x977Lkg3b+fN5g/nbxd+jc5v0fc41M84d3o1Z149iRM8D+N2Ly/jJXz9m0/ZddRqzxE/cEoS7FwNXA28AKwl6Ky03s9vN7JzwtF8Bl5rZYuBp4GIPv4KY2TrgHuBiM9sQpQeUSINXXFLKA3OyGNy1LSce8m016Qn9O3L5Cb156qP1vLpkc53EkpNXyCV/X8DNzy3lsK5teP26UUwY2WNvqaEiXdo25x8/H8kd5x7GJ198zWmT5zL9kw0qTTQC1ljexBEjRrim+5aG5vlPN3DDtMU8+uMjOO3QA79zbE9JKec/Mo81uQXMvOZ4urdvEbc4ZizexO9eXEbhnhJuHjOAi4/pSZMmlSeGaNZ/tZMbn13Mx+u2ceqgzvzvuYPp2DotDhFLbTGzT9x9RLRjiW6kFklaJaXOA3OyGHBga04d2Hmf481SmjBl4nBwuGbqwriMPdi2YzdXPfUp1zy9kF4ZLZl57fH8/LheNUoOAD06tODpy47it2cO5J3MXEZPfoeZS+umBCS1TwlCJEFmLt3M2twdTDq5X4UfyN3bt+DP44awcP127vlPZq2+/psrshk9eS6zlm/hptMOYfoVR9OnY6v9ft6UJsYvju/NzGuOo3v7Fvzy359y7dSFbN+5uxailrqkBCGSAKWlzpTZq+nbqRWnH3ZgpeeeOaQLE0f24OG31zA3c//7cOQV7uHGZxfzi38sIKNVKi9ddRxXndSXpim1+3HQt1NrnrvyGG44tT+vLtnM6MlzmfNZTq2+hsSXEoRIAsxasYXM7AKuPqlvTNU5t541iP6dW3HDtEXk5Nd8goH3s7YyZvJcnv90A1ef1JcZVx/HoIPaVH1hDTVLacI1p/TjxauOpV2LZvzsyfnc8tySej2diHxLCUKkjrk7U2Zn0SujJWcN6RLTNc1TU3jgosPJLyzmV9MWU1rN0cs7dxdz60vL+OETH5HeLIXnrjyGG087hNSmdfMRcFjXtrw86TguP6E3zyz4kjH3zmXemq/q5LWl5pQgROrY7M9yWL4pj1+e2Kda1Tr9O7fm92cfyrurt/Lo3LUxX/fJF9s44753+ce8L/jZsT159ZrjGd7jgJqEvl/Smqbwm9MHMv2Ko0lpYkx8/EP+8PLyhA0GlKo1TXQAUr89u+BLDmybzvH9NJVJea8t3czKLflccUJvWqTG9l/J3bn/rdV0O6A5Pxhefmqyqk0c2Z33s7byf7NWsXJzHlXVTu3aU8J/VmTTpW1znrr0SI7pk/iJCo44uD2vXXs8f37tM/72/jreXpXL0G5tExJLz4yWXHFCH9KbpSTk9StTuKeEx+auZW1uQZXn9ujQkhtO7V/rMShBSIXeXJHNTdOXkNEqlXd/fTLNU+vff6JE+HrHbm6dsZyXF28CYMaijdx9wVCOOLh9ldfOXb2VxRu+4U/nDaZZDRqFzYz/PW8weYV7WLxhe0zXTBzZg1tOH0Dr9GbVfr14aZHalNvHHsboQQfyl9c/Y+GXsd1LbXKHFxdtYsbiTdxzwTCGdW9X5zFUZMmG7dwwbTFZOQX0aN+CKsYqUlQcn+nXNVBOotr8zS7OuO9dmjdLYdM3hfzurEFcclyvRIeVcLM/y+bm55ayfedurj2lH8N7HMDNzy1h0/ZdXDqqN9d/v3+F30bdnfMfmcem7bt4+6aT6qz+Xyr2ftZWbnp2MVvyCvnliX255pR+CX1f9pSUMmV2Fg/OyaJjqzTuHD+EUf3jW3rXQDmplpJS59qpiygqLuWfvziSo3t34NF31iR1XXF+4R5+PX0xP39yAR1apvLiVcdy9cn9OLZvBq9fN4oLv9edR99ZyzkPvMeyjd9EfY55a79iwRdfc8WJfZQc6olj+2bw+vWjGHd4Nx6Yk8XYB99n5ea8hMSyaks+P3jwfe5/azVjhx3EG9ePintyqIr+SmUfU2av5uPPt/HHsYfRp2MrJp3Sl5z8IqYtqD9rFNSlD7K2Mubed5n+yQZ+eWIfXrr6WA496Ns681ZpTfnTeUP428++x/ade/jBg+9z35ur9xn5fP9bq+nUOo0LRnQv/xKSQG3Sm3HX+UN54icjyM0v4pwH3uPBOVkU19GqeSWlziPvrOHsKe+RnVfIoz8+gnsuGEbb5omvElSCkO/4cO1X3P/Was4b3pVxR3QD4OjeHRhx8AE88vYadseprrM+2rW7hNtmLOeiJz4irWkTpl95DL8eM4C0ptGrkE46pBOzrh/FWUO6MPnNTMY9/AGrwzUd5q/bxodrt3F5PW0QFfj+oM785/pRjD70QO56YxXjw3mw4mnd1h1c8Og8/vzaZ5w8oBNvXDdqnzm5EkltELLX1zt2c/p979I8NYWXJx1Hq7Rv+zC8k5nLT//6MX86bzATR/ZIYJR149P1X/OraYv5fOsOLj6mJzePGVCtRvqZSzfz2xeXUVBUzE2jD2Hu6lxWbs5TY38D8fLiTfzupW8nL/zp0TWbvLAipaXOvz76gj/N/IxmKcbtYw9j7LCDqpw5Nx4qa4NQLyYBggbUm6YvZtuO3Tz/02O+kxwARvXLYGj3djz0dhbjj+hWox44AAvXf81Db6+p1eJ705Qm/PDIHpx4SKf9fq6i4hLue3M1j7yzZr+6hp4xuAvf69me/3phKXfMXAnAb06vXpKRxDl76EEc2as9tzy/lD+8vIJZy7O56/whdDtg/2fU3bR9F7+evoT3srZyQv+O/GXcEA5su+96G/WBShACwN/e/5w/vLyC3589iJ8dG7230lsrs7nk7wu4a/wQzq9BPfrWgiJOv+9dSkqdbgc039+Q98rNL2LzN4VcdGQP/uuMgfskt1it2JTHDdMW8dmWfC4c0Z3fnjVwv7uGujvPf7qR2atyuHPcEFrWMDZJDHdn2oIv+eMrQZL/3VkDuWBE9xp903d3nvt0I3+YsZwSd/77zIFcFMN6G/FWWQlCCUJYtvEbznvoA0b1z+Dxn4yo8A/W3Tlrynvs3F3CmzecQEo1itylpc7P/z6fD9Z8xUtXHcvALrU3/0/hnhImv5nJY3PX0u2A5vzf+KEc2btDzNcXl5TyyDtruO+t1bRrkcpfxg3m5AH7Tr8tyevLbTv59fQlzFv7FScP6MSfzxtMpyir7FUkJ7+Q/3p+GW+uzGZkr/b83/ih9OgQv/U9qkPdXKVCBUXFTHp6Ie1bpnLX+KGVfpsxMyad3I/Pt+7glSWbqvU6/++9z3l7VS6/O2tQrSYHgPRmwRQOz15+NE3MmPD4h/zxlRUxdcvNyilg3MMf8H+zMhlzWBdmXTdKyUH20b19C/79iyP5/dmDeD9rK6dOnsuMxbH9H5i5dDOnTZ7L3NW5/PbMgUy99Kh6kxyqohJEkrvhmUW8uGgjT196VEzfuktLPagmcmfWdaNiarhb/OV2xj38Ad8f2JmHf3R4XIvUO3cX8+fXPuMf876gT8eW3HPBMIZGGSFbWur87YN13Pn6Z7RITeGPPziMs4YcFLe4pPFYk1vAr6YtZtGX2zlzSBf+OPYw2rdM3ee87Tt3c+tLy5mxeBNDurXlnguG0rdT6wREXDmVICSq5z7ZwPMLN3LtKf1jrpJp0sS4+uS+ZOUU8NqyLVWen1e4h0lPL6Rzm3T+Mm5I3Otby6Zw+NclR7JzdwnnPfwBd89a9Z3uuV9u28nEsJRxXN8M3rh+lJKDxKxPx1ZMv+JobjrtEGYt38LoyXN5c0X2d86Z81kOoyfPZebSzfzq1P48f+Ux9TI5VEUliCS1JreAs6e8x+CubXnq0qOq1Z5QUuqcOvkdUlOaMPOa4yssRbg7k55eyGvLtjDt8qNimquoNn2zaw+3v7yC5z7dwKAubbj7gqEs+nI7//PKCpqYcevZgxh/RLeENxJKw7Vycx43TFvMys15jD+iG78a3Z/73lzN1Plfckjn1tx9wVAO65qYiQhjpUZq+Y6SUuecB95j0/ZdvHbtqBp1sXth4Qauf2Yxj/34CEZXMLDnmfnrufm5pdx02iFcdVLf/Q27xmYt38J/vbCUrQXBkpfH9OnAXecPpWu72utJJclrd3Ep97+1mofezqLUoYnB5Sf04brv96twUGV9onEQ8h2vLNnE8k153D9xeI37X5895CDufXM1989ezamDOu/zLXx1dj6/n7Gc4/pmcOUJfWoj7BobfeiBjOjZnrtnrWJAlzb8cGSPWh30JMkttWkTbjztEE4Z2Im/vb+Onx5zcJ2XluNFCSLJlJY6D8zOon/nVpw1OLbVzKJpmtKEq07sy6+fW8Lbq3I5acC3g9QK95Rw9VMLaZXWlHsuHFovPozbt0zljnMHJzoMacSG9zggIQsxxZMaqZPM68u3sDqngKtiXAu5Muce3pWu7Zpz/+zVRFZV/vGVFazKzufuC4bRqXX9HCEqIlVTgkgipaXBama9M1rWSq+dZilN+OVJfVi4fjvvZwXrC89cupl/f7Sey0/ozQkJnqpYRPaPEkQSeXNlNp9tyeeqk/pWq9dSZcYf0Y0ubdO5/63VfLltJzc/t4Rh3dtx4+hDauX5RSRxlCCShLszZXYWPdq3YOyw2uvzn9Y0hctH9ebjddu46IkPwWHKxOE1nsxPROoP/S+Og7++9zk3PruYrQVF+/1c7s5TH63nR098RHZeYY2f5+3MXJZu/IarTupD01r+8J4wsgcdW6fx5bZd/HncELq3bxjTCIhI5dSLKQ6e+ng9WTkFzPkshzvOHcyYw2q2AMiWbwq5+bklvJOZC8B1Uxfxr18cWe3qIfeg7aFru+acO7xbjWKpTHqzFO69cBhrt+7gzCE17xklIvWLShC1rKi4hHXhB+WBbdO54l+fcMMzi/hm156Yn8PdeXHhRkZPfoePPv+K28ceyl/GDWbe2q94aE5WtWN6P+srFq7fHte1kI/tm8GPjzo4Ls8tIomhEkQt+3zrDopLndGDOnPG4C48MDuLB+Zk8cGar7hz/JAqFyH/qqCI3764jNeWbeHwHu24+4Jh9MpoibvzwZqvmPxmJkf27sDIXrEPxLl/9moObJPOBSNqv/QgIo2XShC1LDM7WMP2kANb0yylCdef2p8XfnkMrdKb8pO/fsx/v7CUHUXFUa99I5z4662VOdw8ZgDPXnEMvTJaAsFU2//zg8Po3r4F101dyPadu2OK56O1X/Hx59u4/ITeDWLYv4jUH0oQtSxzSz4pTWzvBzvAkG7teGXScVx6fC+e+ng9p9/3Lh9/vm3v8W927eGGaYu4/J+f0LlNOjMmHcuVJ/bZp62hdXozpkwcTm5BEb+evoRY5tGaMjuLjFZpSbGOtIjUrrgmCDMbY2arzCzLzG6JcryHmc0xs4VmtsTMzog49pvwulVmdlo846xNmdn59Mpouc+39fRmKfz3mYN45rKjAbjwsXnc8eoKZn+WzZh75/LSok1cc3JfXrzqWAYcWPGCOkO6tePmMQOYtSKbf374RaWxfPLF17yXtZXLR/UmvZlKDyJSPXFLEGaWAjwInA4MAiaa2aByp/0WmObuw4EJwEPhtYPC7UOBMcBD4fPVe5nZ+fTv3KrC4yN7tee1a4/nopE9ePzdz/n5kwtokZrC81ceww2jD4mpEfmS43px8oBO/M8rK1m+6ZsKz5syezXtW6byw6NUehCR6otnCWIkkOXua919NzAVGFvuHAfKvi63BcrW8BsLTHX3Inf/HMgKn69eK9xTwhfbdtK/c+ULg7RMa8od5w7mX5ccyc1jBvDqNcdHXfWsImbGXeOH0K5FMyY9vTBqm8aSDdt5e1UulxzXixap6osgItUXzwTRFfgyYntDuC/SbcCPzGwDMBOYVI1rMbPLzGyBmS3Izc2trbhrLCunAHc4pIoEUea4fhlceWKfGlX/dGiVxr0ThvH51h38fsbyfY5PmZ1F2+bN+MnR6noqIjWT6EbqicCT7t4NOAP4p5nFHJO7P+buI9x9RMeOiZ8YbtWWfAD6xZgg9tcxfTKYdFJfpn+ygRcWbti7f8WmPP6zIpufH9uL1unN6iQWEWl84pkgNgLdI7a7hfsiXQJMA3D3eUA6kBHjtfVOZk4+qSlN6Nmh7qaauOaUfnyv5wH89oVlfL51BwAPzFlN67SmXHxszzqLQ0Qan3gmiPlAPzPrZWapBI3OM8qdsx44BcDMBhIkiNzwvAlmlmZmvYB+wMdxjLVWZG7Jp3fHlrU+11FlmqY04b4Jw2ma0oRJT3/K8k3f8NqyLVx8bE/aNlfpQURqrspPMjNrYmbDzexMMzvZzDpVdQ2AuxcDVwNvACsJeistN7Pbzeyc8LRfAZea2WLgaeBiDywnKFmsAF4HrnL3kurfXt3KzC7gkAPrpnop0kHtmnPX+CEs25jHxMc+pEWzFH5+bK86j0NEGpcKu7eYWR/gZuD7wGqCb/bpQH8z2wk8Cvzd3Usreg53n0nQ+By579aIxyuAYyu49g7gjpjvJMEKiorZuH0XF3VOTJfS0YceyMXH9OTJD9Zx+Qm9OaBlakLiEJHGo7L+j/8DPAxc7uWG7IaliIuAHwN/j194Dcfq7KCBuqourvH0mzMGMLBLa86shdXiREQqTBDuPrGSYznAvXGJqIHK3JsgKh4kF29pTVO48HsaFCcitaPKEVRmdl6U3d8AS8NEIQTtD+nNmtD9AC2WIyKNQyxDbC8BjgbmhNsnAp8Avczsdnf/Z5xia1Ays/Pp16k1TWpprWcRkUSLpT9mU2Cgu49z93EE8yo5cCRBI7ZQNgdT4tofRERqWywJoru7Z0ds54T7tgGxL5PWiH2zcw/ZeUUJbX8QEaltsVQxvW1mrwDPhtvjwn0tge1xi6wBycwJG6gTMAZCRCReYkkQVxEkhbLxCv8Angu7vp4Ur8AakrI5mFTFJCKNSZUJIkwE08MfiWJ1dj6t0ppyUNv0RIciIlJrYplq4ygzm29mBWa228xKzCyvLoJrKFZl59OvcyvM1INJRBqPWBqpHyCYlns10Bz4BcFKcRJanV0Q8xoQIiINRUzTjrp7FpDi7iXu/jeCZUAF2FpQxFc7dtfZGhAiInUllkbqneF03YvM7E5gM4lfaKjeKJtiQyUIEWlsYvmg/3F43tXADoKFfMbFM6iGJHNL4udgEhGJh1h6MX0RliB6As8Dq9x9d7wDaygycwpo16IZHVunJToUEZFaFctkfWcCjwBrACOYg+lyd38t3sE1BJlbgik21INJRBqbWNog7gZOChuqyxYSehVI+gTh7mRm53POMK2/ICKNTyxtEPllySG0FsiPUzwNSnZeEXmFxWqgFpFGqbIlR8vWgVhgZjMJ1oh24Hxgfh3EVu+V9WBSF1cRaYwqq2I6O+JxNnBC+Lhsbeqkl1kPlhkVEYmXypYc/VldBtIQZWbnk9EqjfYtUxMdiohIravWgDcz+zRegTREq7ILOORAjX8QkcapuiOi1ZczVFrqZIXLjIqINEYVJggzuzb899iI3a/GPaIGYuP2XezYXcIhWiRIRBqpykoQZW0QU8p2uPtv4xtOw/FtA7WqmESkcaqsF9NKM1sNHGRmSyL2G8E6QkPiG1r9lpldAKiLq4g0XpX1YppoZgcCbwDn1F1IDUNmdj5d2qbTJr1ZokMREYmLSqfacPctwNBwsr7+4e5V7r4n7pHVc5nZ+Rr/ICKNWixLjp5AsJrcg8BDQKaZjYp3YPVZSamTlVOg9gcRadRimazvHmC0u68CMLP+wNPAEfEMrD5bv20nRcWlKkGISKMWyziIZmXJAcDdM4GkrnhftUVTbIhI4xdLCWKBmT0B/Cvc/iGwIH4h1X+r907SpyomEWm8YkkQVwJXAdeE2+8StEUkrVXZ+XRv35wWqbH8+kREGqYqq5jcvcjd73H384Cr3H2yuxfF8uRmNsbMVplZlpndEuX4ZDNbFP5kmtn2iGN/MbNl4c+F1bqrOFudXaA1IESk0avuV+BXgcNjOdHMUgh6Pp0KbADmm9kMd19Rdo67Xx9x/iRgePj4zPB1hgFpwNtm9pq751Uz3lq3p6SUtVsLOHlgp0SHIiISV/GcrG8kkOXua919NzAVGFvJ+RMJekcBDALmunuxu+8AlgBjqhlrXKzbuoM9Ja4ShIg0etVNEI9X49yuwJcR2xvCffsws4OBXsDscNdiYIyZtTCzDOAkoHs1Y42LVVokSESSRCwD5f5Z9tjdHyq/r5ZMAKa7e0n4OrOAmcAHBKWKeUBJlNguM7MFZrYgNze3lkOKLjO7gCYGvTu2rJPXExFJlFhKEIdGboRtC7EMktvId7/1dwv3RTOBb6uXAHD3O9x9mLufSlC1lVn+Ind/zN1HuPuIjh07xhDS/svckk/PjJakN0upk9cTEUmUytaD+I2Z5QNDzCwv/MkHcoCXYnju+UA/M+sVzuU0AZgR5XUGAAcQlBLK9qWYWYfw8RBgCDCrGvcVN5k5+fTXIkEikgQqTBDu/id3bw3c5e5twp/W7t7B3X9T1RO7ezFwNcFssCuBae6+3MxuN7PI2WEnAFPd3SP2NQPeNbMVwGPAj8LnS6jCPSWs27qD/lokSESSQCzdXF+LNjmfu8+t6kJ3n0nQlhC579Zy27dFua6QoCdTvbI2dwelrkWCRCQ5xJIgbop4nE7QffUT4OS4RFSPla0ipy6uIpIMqkwQ7n525LaZdQfujVtE9Vhmdj7NUoyeGerBJCKNX3XHQUAwnmFgbQfSEGRm59M7oxXNUmryaxMRaViqLEGY2RSgrAG5CcH0F5/GM6j6anVOAYO7tk10GCIidSKm6b4jHhcDT7v7+3GKp95ydzZt38UZg7skOhQRkToRSxvE38uvSR3fkOqnr3fuYU+J06l1WqJDERGpE7FUMZ0I/B1YRzCiubuZ/TSWbq6NSU5+IQCdWqcnOBIRkboRSxXT3WhNanLygiUwOrVRCUJEkoPWpI5RTn6QIDqrBCEiSUJrUscoOy+sYlIJQkSShNakjlFufhGt05tqFlcRSRqx9GIqAu4Jf5JWTn6hejCJSFKpbLrvl83sbDPbp73BzHqHs7L+PL7h1R85eUV0bqP2BxFJHpWVIC4FbgDuNbNtQC7BZH29gCzgAXePZV2IRiE7v5AjehyQ6DBEROpMhQnC3bcAvwZ+bWY9gS7ALiDT3XfWSXT1hLuTk1dEJ5UgRCSJxNJIjbuvIxgol5TyCospKi5VG4SIJBVNSxqDnL1dXFWCEJHkoQQRg7JBcipBiEgyqTJBhD2ZkjqRfDsPkxKEiCSPWD74LwRWm9mdZjYg3gHVR9/Ow6QqJhFJHlUmCHf/ETAcWAM8aWbzzOwyM0uahZmz84pomZpCq7SY2vRFRBqFmKqO3D0PmA5MJejuei7wqZlNimNs9UZOfqFKDyKSdGJpgzjHzF4A3iaYxXWku58ODAV+Fd/w6oec/CI6qv1BRJJMLHUm44DJ5RcIcvedZnZJfMKqX3LyChncrV2iwxARqVOxJIjbgM1lG2bWHOjs7uvc/a14BVaf5OQXqQeTiCSdWNogngVKI7ZLwn1JoaComJ27S5QgRCTpxJIgmrr77rKN8HFq/EKqX3K0UJCIJKlYEkSumZ1TtmFmY4Gt8QupfsnO01KjIpKcYmmDuAL4t5k9ABjwJfCTuEZVj+wdRa0ShIgkmVhWlFsDHGVmrcLtgrhHVY/khvMwdVQJQkSSTExDg83sTOBQIN3MAHD32+MYV72Rk19EWtMmtEnXKGoRSS6xDJR7hGA+pkkEVUznAwfHOa56IzuvkM5t0ilLjCIiySKWRupj3P0nwNfu/gfgaKB/fMOqP3LyNAZCRJJTLAmiMPx3p5kdBOwhmI8pKQTzMClBiEjyiSVBvGxm7YC7gE8Jlh59KpYnN7MxZrbKzLLM7JYoxyeb2aLwJ9PMtiL8CdcAAA+WSURBVEccu9PMlpvZSjO73xJUxxOUINRALSLJp9KW13ChoLfcfTvwnJm9AqS7+zdVPbGZpQAPAqcCG4D5ZjbD3VeUnePu10ecP4lgWnHM7BjgWGBIePg94ASCCQPrzK7dJeQXFasEISJJqdIShLuXEnzIl20XxZIcQiOBLHdfG46+ngqMreT8icDTZS8FpBOM2E4jmEU2O8bXrTXfriSnEoSIJJ9YqpjeMrNxNaji6UowqK7MhnDfPszsYKAXMBvA3ecBcwgmCdwMvOHuK6Ncd5mZLTCzBbm5udUMr2pai1pEklksCeJygsn5iswsz8zyzSyvluOYAEx39xIAM+sLDAS6ESSVk83s+PIXuftj7j7C3Ud07NixlkMKurgCdNZiQSKShGJZcrS1uzdx91R3bxNut4nhuTcC3SO2u4X7opnAt9VLEKxY96G7F4Qjt18j6F5bp/auRa0ShIgkoSqHB5vZqGj7yy8gFMV8oJ+Z9SJIDBOAi6I8/wDgAGBexO71wKVm9ieCwXknAPdWFWtty8kvIjWlCe1aNKvrlxYRSbhY5o+4KeJxOkHj8yfAyZVd5O7FZnY18AaQAvzV3Zeb2e3AAnefEZ46AZjq7h5x+fTw+ZcSNFi/7u4vx3JDtSknv5COrdM0ilpEklIsk/WdHbltZt2J8du8u88EZpbbd2u57duiXFdC0PaRUDl5ReriKiJJK5ZG6vI2EDQgN3o5+YVqfxCRpBVLG8QUgmoeCBLKMIIR1Y1eTn4RR/bqkOgwREQSIpY2iAURj4uBp939/TjFU28U7ilh+849dFYVk4gkqVgSxHSgMGKMQoqZtXD3nfENLbFy9w6S0xgIEUlOMY2kBppHbDcH3oxPOPVH2SjqjipBiEiSiiVBpEcuMxo+bhG/kOqH3L3zMClBiEhyiiVB7DCzw8s2zOwIYFf8QqofssNR1JpmQ0SSVSxtENcBz5rZJoJRzQcSLEHaqOXkF9K0idG+RWqiQxERSYhYBsrND6fDOCTctcrd98Q3rMTLySsio1UaTZpoFLWIJKcqq5jM7Cqgpbsvc/dlQCsz+2X8Q0usnHyNohaR5BZLG8Sl4YpyALj718Cl8QupfsjOK1QXVxFJarEkiJTIxYLCpUQbfcV8rkoQIpLkYmmkfh14xsweDbcvD/c1WntKSvlqx251cRWRpBZLgrgZuAy4Mtz+D/B43CKqBzSKWkQkthXlSt39EXcf7+7jgRXAlPiHljhlo6g1D5OIJLNYShCY2XBgInAB8DnwfDyDSrScvLJR1CpBiEjyqjBBmFl/gqQwEdgKPAOYu59UR7ElTFkJQo3UIpLMKitBfAa8C5zl7lkAZnZ9nUSVYDl5hTQx6NCy0XfWEhGpUGVtEOcBm4E5Zva4mZ1CMNVGo5eTX0SHVmk0TanJgnsiIo1DhZ+A7v6iu08ABgBzCOZk6mRmD5vZ6LoKMBFy8ovUxVVEkl4svZh2uPtT7n420A1YSND1tdEKRlErQYhIcqtWHYq7f+3uj7n7KfEKqD7IyS/SNN8ikvRUyV5OSanzVYGqmERElCDK+aqgiFKHjipBiEiSU4Iop2wlOZUgRCTZKUGUkxOuRa02CBFJdkoQ5ewdRa0ShIgkOSWIcnLCKqaMVkoQIpLclCDKyc4vpH3LVFKb6lcjIslNn4Ll5OSpi6uICChB7CM3v5BOaqAWEVGCKC9bJQgREUAJ4jtKS52tBUVaSU5EhDgnCDMbY2arzCzLzG6JcnyymS0KfzLNbHu4/6SI/YvMrNDMfhDPWAG27dxNcalrJTkREWJccrQmzCwFeBA4FdgAzDezGe6+ouwcd78+4vxJwPBw/xxgWLi/PZAFzIpXrGVyNIpaRGSveJYgRgJZ7r7W3XcDU4GxlZw/EXg6yv7xwGvuvjMOMX5HdjiKWkuNiojEN0F0Bb6M2N4Q7tuHmR0M9AJmRzk8geiJAzO7zMwWmNmC3Nzc/QwXcveWIFTFJCJSXxqpJwDT3b0kcqeZdQEGA29Euyhcm2KEu4/o2LHjfgdRNg9TR1UxiYjENUFsBLpHbHcL90VTUSnhAuAFd99Ty7FFlZNfRNvmzUhvllIXLyciUq/FM0HMB/qZWS8zSyVIAjPKn2RmA4ADgHlRnqOidom40FKjIiLfiluCcPdi4GqC6qGVwDR3X25mt5vZORGnTgCmurtHXm9mPQlKIO/EK8bytNSoiMi34tbNFcDdZwIzy+27tdz2bRVcu44KGrXjJSeviCN7tazLlxQRqbfqSyN1wrk7uflFdFQXVxERQAlir+0797C7pFRdXEVEQkoQobKV5DQPk4hIQAkiVDYGQiUIEZGAEkQoW/MwiYh8hxJEKEfzMImIfIcSRCgnr4jWaU1pkRrXnr8iIg2GEkRIXVxFRL5LCSKkaTZERL5LCSKkaTZERL5LCYJgFHVOvkoQIiKRlCCAvMJiCvdoFLWISCQlCCBXXVxFRPahBEHQxRU0ilpEJJISBN/Ow6QShIjIt5QgCLq4gqbZEBGJpARBUIJokZpCqzSNohYRKaMEQZAgOrVOw8wSHYqISL2hBAHk5BWqgVpEpBwlCIIShOZhEhH5LiUIghJEZ5UgRES+I+kTREFRMTt2l6iLq4hIOUmfIHYXl3L20IMY1KVNokMREalXkr5fZ/uWqUyZODzRYYiI1DtJX4IQEZHolCBERCQqJQgREYlKCUJERKJSghARkaiUIEREJColCBERiUoJQkREojJ3T3QMtcLMcoEvanh5BrC1FsNJpMZyL43lPkD3Ul/pXgIHu3vHaAcaTYLYH2a2wN1HJDqO2tBY7qWx3AfoXuor3UvVVMUkIiJRKUGIiEhUShCBxxIdQC1qLPfSWO4DdC/1le6lCmqDEBGRqFSCEBGRqJQgREQkqqROEGY2xsxWmVmWmd2S6Hiqy8zWmdlSM1tkZgvCfe3N7D9mtjr894BExxmNmf3VzHLMbFnEvqixW+D+8H1aYmaHJy7yfVVwL7eZ2cbwvVlkZmdEHPtNeC+rzOy0xEQdnZl1N7M5ZrbCzJab2bXh/gb13lRyHw3ufTGzdDP72MwWh/fyh3B/LzP7KIz5GTNLDfenhdtZ4fGeNX5xd0/KHyAFWAP0BlKBxcCgRMdVzXtYB2SU23cncEv4+BbgL4mOs4LYRwGHA8uqih04A3gNMOAo4KNExx/DvdwG3Bjl3EHh31oa0Cv8G0xJ9D1ExNcFODx83BrIDGNuUO9NJffR4N6X8HfbKnzcDPgo/F1PAyaE+x8Brgwf/xJ4JHw8AXimpq+dzCWIkUCWu691993AVGBsgmOqDWOBv4eP/w78IIGxVMjd5wLbyu2uKPaxwD888CHQzsy61E2kVavgXioyFpjq7kXu/jmQRfC3WC+4+2Z3/zR8nA+sBLrSwN6bSu6jIvX2fQl/twXhZrPwx4GTgenh/vLvSdl7NR04xcysJq+dzAmiK/BlxPYGKv8Dqo8cmGVmn5jZZeG+zu6+OXy8BeicmNBqpKLYG+p7dXVY7fLXiKq+BnMvYdXEcIJvrA32vSl3H9AA3xczSzGzRUAO8B+CEs52dy8OT4mMd++9hMe/ATrU5HWTOUE0Bse5++HA6cBVZjYq8qAHZcwG2Y+5IcceehjoAwwDNgN3Jzac6jGzVsBzwHXunhd5rCG9N1Huo0G+L+5e4u7DgG4EJZsBdfG6yZwgNgLdI7a7hfsaDHffGP6bA7xA8IeTXVbED//NSVyE1VZR7A3uvXL37PA/dSnwON9WV9T7ezGzZgQfqv929+fD3Q3uvYl2Hw35fQFw9+3AHOBoguq8puGhyHj33kt4vC3wVU1eL5kTxHygX9gTIJWgMWdGgmOKmZm1NLPWZY+B0cAygnv4aXjaT4GXEhNhjVQU+wzgJ2GPmaOAbyKqO+qlcvXw5xK8NxDcy4Swp0kvoB/wcV3HV5Gwrvr/ASvd/Z6IQw3qvanoPhri+2JmHc2sXfi4OXAqQZvKHGB8eFr596TsvRoPzA5LfdWX6Bb6RP4Q9MDIJKjP++9Ex1PN2HsT9LpYDCwvi5+grvEtYDXwJtA+0bFWEP/TBEX8PQT1p5dUFDtBL44Hw/dpKTAi0fHHcC//DGNdEv6H7RJx/n+H97IKOD3R8Ze7l+MIqo+WAIvCnzMa2ntTyX00uPcFGAIsDGNeBtwa7u9NkMSygGeBtHB/eridFR7vXdPX1lQbIiISVTJXMYmISCWUIEREJColCBERiUoJQkREolKCEBGRqJQgpFEwMzezuyO2bzSz22rpuZ80s/FVn7nfr3O+ma00sznl9vc0s4vi/foi5SlBSGNRBJxnZhmJDiRSxEjXWFwCXOruJ5Xb3xOImiCq+fwi1aIEIY1FMcG6vNeXP1C+BGBmBeG/J5rZO2b2kpmtNbM/m9kPw7n3l5pZn4in+b6ZLTCzTDM7K7w+xczuMrP54eRvl0c877tmNgNYESWeieHzLzOzv4T7biUY3PX/zOyucpf8GTg+XL/gejO72MxmmNls4K1wVP1fw7gXmtnYKuLrYmZzw+dbZmbH1/B3Lo2cvn1IY/IgsMTM7qzGNUOBgQTTda8FnnD3kRYsMDMJuC48ryfBvD19gDlm1hf4CcHUEt8zszTgfTObFZ5/OHCYB1NH72VmBwF/AY4AviaYjfcH7n67mZ1MsFbBgnIx3hLuL0tMF4fPP8Tdt5nZ/xJMp/DzcEqGj83sTeCHFcR3HvCGu99hZilAi2r8viSJKEFIo+HueWb2D+AaYFeMl833cO4gM1sDlH3ALwUiq3qmeTDB22ozW0swm+ZoYEhE6aQtwRw+u4GPyyeH0PeAt909N3zNfxMsOPRijPGW+Y+7l61BMRo4x8xuDLfTgR6VxDcf+Gs4md2L7r6omq8tSUIJQhqbe4FPgb9F7CsmrE41syYEKwiWKYp4XBqxXcp3/3+Un5PGCeYhmuTub0QeMLMTgR01Cz9mkc9vwDh3X1UujqjxhcdGAWcCT5rZPe7+j7hGKw2S2iCkUQm/VU8jaPAts46gSgfgHIIVuarrfDNrErZL9CaY0O0N4Mrwmzhm1j+cWbcyHwMnmFlGWL0zEXinimvyCZbNrMgbwKQwIWBmwyP27xOfmR0MZLv748ATBNVVIvtQCUIao7uBqyO2HwdeMrPFwOvU7Nv9eoIP9zbAFe5eaGZPELRNfBp+OOdSxRKv7r7ZzG4hmKrZgFfdvaop2ZcAJWH8TxK0XUT6I0HJaUlYQvocOIvgwz9afCcCN5nZHqCAoC1FZB+azVVERKJSFZOIiESlBCEiIlEpQYiISFRKECIiEpUShIiIRKUEISIiUSlBiIhIVP8f9FToE3FLDtYAAAAASUVORK5CYII=", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "import matplotlib.pyplot as plt\n", + "logs = model.make_inspector().training_logs()\n", + "plt.plot([log.num_trees for log in logs], [log.evaluation.accuracy for log in logs])\n", + "plt.xlabel(\"Number of trees\")\n", + "plt.ylabel(\"Accuracy (out-of-bag)\")\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "KWlw6i0U9UcE" + }, + "source": [ + "You can also see some general stats on the OOB dataset:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "_nEjaF9Y9NjF" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "Evaluation(num_examples=611, accuracy=0.806873977086743, loss=0.7393123309627944, rmse=None, ndcg=None, aucs=None, auuc=None, qini=None)" + ] + }, + "execution_count": 18, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "inspector = model.make_inspector()\n", + "inspector.evaluation()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "W8OwVx569bbU" + }, + "source": [ + "Now, let's run an evaluation using the test data." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "KyH_XC1d9X9x" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "1/1 [==============================] - 1s 958ms/step - loss: 0.0000e+00 - accuracy: 0.8679\n", + "loss: 0.0000\n", + "accuracy: 0.8679\n" + ] + } + ], + "source": [ + "evaluation = model.evaluate(x=val_ds,return_dict=True)\n", + "\n", + "for name, value in evaluation.items():\n", + " print(f\"{name}: {value:.4f}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "TK0l4Qgxbwcq" + }, + "source": [ + "## Test Set Prediction\n", + "Now we will do prediction on `test.csv`.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "BbC05cBKcTQ5" + }, + "outputs": [], + "source": [ + "test_file_path = os.path.join(DOWNLOAD_LOCATION, \"test.csv\")\n", + "test_data = pd.read_csv(test_file_path)\n", + "ids = test_data.pop('PassengerId')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "OUDeYu2zcYrk" + }, + "outputs": [], + "source": [ + "test_ds = tfdf.keras.pd_dataframe_to_tf_dataset(\n", + " test_data, \n", + " task = tfdf.keras.Task.CLASSIFICATION)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "4l13fn1seaHj" + }, + "source": [ + "Since the prediction can be either 0 (Not survived) or 1 (Survived), let's convert the predited float value to binary value" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "8tY1evHAcoTH" + }, + "outputs": [], + "source": [ + "preds = model.predict(test_ds)\n", + "preds = preds >= 0.5\n", + "preds = preds.astype('int')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "Jxtj1lp6csVQ" + }, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "
\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
PassengerIdSurvived
08920
18930
28940
38950
48960
\n", + "
\n", + " \n", + " \n", + " \n", + "\n", + " \n", + "
\n", + "
\n", + " " + ], + "text/plain": [ + " PassengerId Survived\n", + "0 892 0\n", + "1 893 0\n", + "2 894 0\n", + "3 895 0\n", + "4 896 0" + ] + }, + "execution_count": 23, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "output = pd.DataFrame({'PassengerId': ids,\n", + " 'Survived': preds.squeeze()})\n", + "\n", + "output.head()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "yZOhWNRcpM-G" + }, + "source": [ + "You can download the predicted output as a CSV file and do submission on the [Competition page](https://www.kaggle.com/competitions/titanic/submit) on Kaggle." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "j8FquAuufmtS" + }, + "outputs": [], + "source": [ + "output_filename = \"test_prediction_output.csv\"\n", + "output.to_csv(output_filename, index=False)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "2ary3LNoffRA" + }, + "outputs": [ + { + "data": { + "application/javascript": [ + "\n", + " async function download(id, filename, size) {\n", + " if (!google.colab.kernel.accessAllowed) {\n", + " return;\n", + " }\n", + " const div = document.createElement('div');\n", + " const label = document.createElement('label');\n", + " label.textContent = `Downloading \"${filename}\": `;\n", + " div.appendChild(label);\n", + " const progress = document.createElement('progress');\n", + " progress.max = size;\n", + " div.appendChild(progress);\n", + " document.body.appendChild(div);\n", + "\n", + " const buffers = [];\n", + " let downloaded = 0;\n", + "\n", + " const channel = await google.colab.kernel.comms.open(id);\n", + " // Send a message to notify the kernel that we're ready.\n", + " channel.send({})\n", + "\n", + " for await (const message of channel.messages) {\n", + " // Send a message to notify the kernel that we're ready.\n", + " channel.send({})\n", + " if (message.buffers) {\n", + " for (const buffer of message.buffers) {\n", + " buffers.push(buffer);\n", + " downloaded += buffer.byteLength;\n", + " progress.value = downloaded;\n", + " }\n", + " }\n", + " }\n", + " const blob = new Blob(buffers, {type: 'application/binary'});\n", + " const a = document.createElement('a');\n", + " a.href = window.URL.createObjectURL(blob);\n", + " a.download = filename;\n", + " div.appendChild(a);\n", + " a.click();\n", + " div.remove();\n", + " }\n", + " " + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "application/javascript": [ + "download(\"download_1a8bee3f-c06f-4c3e-8d5a-94a8e2980331\", \"test_prediction_output.csv\", 2839)" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "from google.colab import files\n", + "files.download('test_prediction_output.csv')" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Lh3gxL9OHKbD" + }, + "source": [ + "# References\n", + "* Dive deep into \n", + " * [Random Forests](https://www.tensorflow.org/decision_forests/api_docs/python/tfdf/keras/RandomForestModel)\n", + " * [Gradient Boosted Trees](https://www.tensorflow.org/decision_forests/api_docs/python/tfdf/keras/GradientBoostedTreesModel)\n", + " * [CART](https://www.tensorflow.org/decision_forests/api_docs/python/tfdf/keras/CartModel)\n", + " * [Keras API](https://www.tensorflow.org/api_docs/python/tf/keras)\n", + " * [TensorFlow Decision Forests (TF-DF)](https://www.tensorflow.org/decision_forests).\n", + "* [EDA is fun](https://www.kaggle.com/code/prashant111/eda-is-fun#EDA-is-fun) by Prashant Banerjee.\n", + "* TensorFlow Decision Forests tutorials which are a set of 3 very interesting tutorials.\n", + " * [Beginner Tutorial](https://www.tensorflow.org/decision_forests/tutorials/beginner_colab)\n", + " * [Intermediate Tutorial](https://www.tensorflow.org/decision_forests/tutorials/intermediate_colab)\n", + " * [Advanced Tutorial](https://www.tensorflow.org/decision_forests/tutorials/advanced_colab)\n", + "* The [TensorFlow Forum](https://discuss.tensorflow.org/) where one can get in touch with the TensorFlow community. Check it out if you haven't yet." + ] + } + ], + "metadata": { + "colab": { + "collapsed_sections": [], + "name": "kaggle_beginner_example_classification.ipynb", + "toc_visible": true + }, + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.9" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +}