Day_2_Lab_3_RNNs.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "gpuType": "T4",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/DrKenReid/Introductory-Data-Science/blob/main/Day_2_Lab_3_RNNs.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Recurrent Neural Networks\n",
        "\n",
        "To use GPU:\n",
        "\n",
        "Go to `Runtime` in the top menu bar, select `Change runtime type`, and the select `T4 GPU`. This will significantly decrease the running time of model training."
      ],
      "metadata": {
        "id": "MJPX_E8e0_4x"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Problem/dataset Description:\n",
        "\n",
        "For this hands-on exercise, we will work with the IMDB Reviews dataset and perform sentiment classification. We will use a simple RNN, LSTM, and GRU as our classification models. The task is to classify each movie review into one of two categories (positive or negative).\n",
        "\n",
        "The steps involve: preprocessing the text data, feeding it into one of the neural network models, and training the model to predict the sentiment based on the textual content of the reviews.\n",
        "\n",
        "### Explanation of the Dataset Installation and Q&A\n",
        "\n",
        "1. **Q: What does `!pip install datasets` do?**\n",
        "   - **A: It installs the `datasets` library, which is commonly used for accessing and managing a wide variety of datasets in machine learning.**\n",
        "\n",
        "### Likely Student Questions and Answers\n",
        "\n",
        "**Q: Why do we use `!pip install datasets` in the code?**\n",
        "- **A: The exclamation mark `!` allows us to run shell commands from within a Jupyter notebook or other interactive Python environments. `pip install datasets` installs the `datasets` library, which provides easy access to many machine learning datasets.**\n",
        "\n",
        "**Q: What is the `datasets` library used for?**\n",
        "- **A: The `datasets` library, developed by Hugging Face, provides a collection of ready-to-use datasets for various machine learning tasks, along with tools for loading, processing, and manipulating them efficiently.**\n",
        "\n",
        "**Q: Why might we prefer using the `datasets` library over other dataset libraries?**\n",
        "- **A: The `datasets` library is designed for high performance, scalability, and ease of use. It supports a wide range of datasets, integrates well with popular ML frameworks like PyTorch and TensorFlow, and offers built-in functions for common data preprocessing tasks.**\n",
        "\n",
        "**Q: Can I use `!pip install datasets` in any Python environment?**\n",
        "- **A: Yes, you can use this command in any environment where `pip` is available, such as Jupyter notebooks, command-line interfaces, and some IDEs that support shell commands.**\n",
        "\n",
        "**Q: What should I do if the installation of `datasets` fails?**\n",
        "- **A: Ensure that you have an active internet connection and the correct permissions to install packages. If the issue persists, check for specific error messages and consult the `datasets` library documentation or community forums for troubleshooting tips.**\n",
        "\n",
        "**Q: How do I verify that the `datasets` library has been installed correctly?**\n",
        "- **A: You can verify the installation by importing the library in your Python environment with `import datasets`. If there are no errors, the installation was successful.**\n",
        "\n",
        "**Q: Is it necessary to install the `datasets` library every time I run my code?**\n",
        "- **A: No, once installed, the library remains available in your Python environment. You only need to install it again if you switch to a different environment or if the library is updated and you want to install the latest version.**\n"
      ],
      "metadata": {
        "id": "2lJPetMoaj3F"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install datasets"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zut0hsM6CEU3",
        "outputId": "e762b699-6069-40ca-cb61-2d4eeaa21a09"
      },
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Requirement already satisfied: datasets in /usr/local/lib/python3.10/dist-packages (2.21.0)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from datasets) (3.15.4)\n",
            "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from datasets) (1.26.4)\n",
            "Requirement already satisfied: pyarrow>=15.0.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (17.0.0)\n",
            "Requirement already satisfied: dill<0.3.9,>=0.3.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (0.3.8)\n",
            "Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from datasets) (2.1.4)\n",
            "Requirement already satisfied: requests>=2.32.2 in /usr/local/lib/python3.10/dist-packages (from datasets) (2.32.3)\n",
            "Requirement already satisfied: tqdm>=4.66.3 in /usr/local/lib/python3.10/dist-packages (from datasets) (4.66.5)\n",
            "Requirement already satisfied: xxhash in /usr/local/lib/python3.10/dist-packages (from datasets) (3.5.0)\n",
            "Requirement already satisfied: multiprocess in /usr/local/lib/python3.10/dist-packages (from datasets) (0.70.16)\n",
            "Requirement already satisfied: fsspec<=2024.6.1,>=2023.1.0 in /usr/local/lib/python3.10/dist-packages (from fsspec[http]<=2024.6.1,>=2023.1.0->datasets) (2024.6.1)\n",
            "Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from datasets) (3.10.3)\n",
            "Requirement already satisfied: huggingface-hub>=0.21.2 in /usr/local/lib/python3.10/dist-packages (from datasets) (0.23.5)\n",
            "Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from datasets) (24.1)\n",
            "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from datasets) (6.0.2)\n",
            "Requirement already satisfied: aiohappyeyeballs>=2.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (2.3.5)\n",
            "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.3.1)\n",
            "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (24.2.0)\n",
            "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.4.1)\n",
            "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (6.0.5)\n",
            "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.9.4)\n",
            "Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (4.0.3)\n",
            "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.21.2->datasets) (4.12.2)\n",
            "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests>=2.32.2->datasets) (3.3.2)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests>=2.32.2->datasets) (3.7)\n",
            "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.32.2->datasets) (2.0.7)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests>=2.32.2->datasets) (2024.7.4)\n",
            "Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets) (2.8.2)\n",
            "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets) (2024.1)\n",
            "Requirement already satisfied: tzdata>=2022.1 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets) (2024.1)\n",
            "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.2->pandas->datasets) (1.16.0)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import torch\n",
        "import torch.nn as nn\n",
        "from torch.utils.data import Dataset, DataLoader\n",
        "from transformers import BertTokenizer\n",
        "from sklearn.model_selection import train_test_split\n",
        "from datasets import load_dataset\n",
        "import numpy as np"
      ],
      "metadata": {
        "id": "XEcVdc8YWFeq"
      },
      "execution_count": 14,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Likely Student Questions and Answers\n",
        "\n",
        "**Q: Why is the `MAX_LEN` set to 256 tokens?**\n",
        "- **A: This value is chosen based on the typical length of the input data. It helps to ensure that the sequences are not too long, which could increase computation time and memory usage, while still capturing enough information for the model to learn effectively.**\n",
        "\n",
        "**Q: What is the significance of setting the `BATCH_SIZE` to 16?**\n",
        "- **A: A batch size of 16 is a trade-off between memory efficiency and training speed. Smaller batches can make the model update more frequently but may not fully utilize GPU capabilities, while larger batches can lead to more stable updates but require more memory.**\n",
        "\n",
        "**Q: Why do we train the model for only 3 epochs?**\n",
        "- **A: Training for 3 epochs is often a starting point to avoid overfitting and to quickly evaluate the model's performance. Depending on the results, you might increase or decrease the number of epochs.**\n",
        "\n",
        "**Q: How does the `LEARNING_RATE` affect model training?**\n",
        "- **A: The learning rate controls how much the model's weights are adjusted with respect to the loss gradient. A smaller learning rate can lead to more precise convergence but slower training, while a larger learning rate can speed up training but might overshoot the optimal solution.**\n",
        "\n",
        "**Q: Why do we check for `cuda` availability with `torch.cuda.is_available()`?**\n",
        "- **A: Checking for `cuda` availability ensures that the model can utilize a GPU if available, which significantly speeds up training and evaluation compared to using a CPU.**\n",
        "\n",
        "**Q: How do we decide the appropriate `LEARNING_RATE` for a model?**\n",
        "- **A: The appropriate learning rate is often found through experimentation and hyperparameter tuning. Techniques like learning rate schedules or adaptive learning rate methods (e.g., Adam optimizer) can also help in finding an effective learning rate.**\n",
        "\n",
        "**Q: What if the sequences in my data are longer than `MAX_LEN`?**\n",
        "- **A: Sequences longer than `MAX_LEN` will be truncated, which might lead to loss of information. If this is a concern, consider increasing `MAX_LEN` or using techniques like sequence padding and truncation to manage sequence lengths appropriately.**\n",
        "\n",
        "**Q: Why is it beneficial to use a GPU for training neural networks?**\n",
        "- **A: GPUs are designed for parallel processing, making them much faster than CPUs for the large-scale computations required in training neural networks. This results in quicker training times and the ability to handle larger datasets and models.**\n"
      ],
      "metadata": {
        "id": "KkR32MK-X8YV"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "MAX_LEN = 256  # Maximum length of tokens\n",
        "BATCH_SIZE = 16  # Number of samples per batch\n",
        "EPOCHS = 3  # Number of complete passes through the dataset\n",
        "LEARNING_RATE = 1e-3  # Step size for updating model weights\n",
        "DEVICE = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")  # Use GPU if available, otherwise use CPU"
      ],
      "metadata": {
        "id": "nmsy5qyXyC6l"
      },
      "execution_count": 15,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Data preparation"
      ],
      "metadata": {
        "id": "jZRvRBgDao67"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "The IMDB dataset is a collection of movie reviews, each tagged with a sentiment label: positive or negative. We will do binary classification on this dataset.\n",
        "\n",
        "For the purpose of illustrations in this lab, we are using a portion of the full IMDB dataset.\n",
        "\n",
        "### Explanation of the IMDBDataset Class and Q&A\n",
        "\n",
        "3. **Q: Why do we use `BertTokenizer.from_pretrained('bert-base-uncased')`?**\n",
        "   - **A: To load the pretrained BERT tokenizer, which is used to convert text into tokens that BERT can understand.**\n",
        "\n",
        "4. **Q: What does the `__len__` method do?**\n",
        "   - **A: It returns the number of samples in the dataset, which is the length of the texts list.**\n",
        "\n",
        "5. **Q: What is the purpose of the `__getitem__` method?**\n",
        "   - **A: It retrieves a single sample from the dataset at the given index, tokenizes the text, and returns the tokenized input along with the attention mask and label.**\n",
        "\n",
        "6. **Q: How is the text tokenized in the `__getitem__` method?**\n",
        "   - **A: Using the `tokenizer.encode_plus` method, which tokenizes the text, adds special tokens, and returns token IDs, attention masks, and handles padding and truncation.**\n",
        "\n",
        "7. **Q: What does `padding='max_length'` do in the `encode_plus` method?**\n",
        "   - **A: It pads the token sequence to the maximum length specified by `MAX_LEN`.**\n",
        "\n",
        "8. **Q: Why do we use `return_attention_mask=True` in `encode_plus`?**\n",
        "   - **A: To generate an attention mask, which indicates which tokens are actual data and which are padding, helping the model to ignore the padding tokens.**\n",
        "\n",
        "9. **Q: What is the significance of `return_tensors='pt'` in the tokenizer encoding?**\n",
        "   - **A: It ensures that the tokenized output is returned as PyTorch tensors, suitable for input into a PyTorch model.**\n",
        "\n",
        "10. **Q: Why do we flatten the `input_ids` and `attention_mask` tensors?**\n",
        "    - **A: To convert them from a 2D tensor (batch size, sequence length) to a 1D tensor, as expected by the model.**\n",
        "\n",
        "11. **Q: How is the label converted to a tensor?**\n",
        "    - **A: Using `torch.tensor(label, dtype=torch.long)` to convert the label into a long tensor, suitable for loss calculation in PyTorch.**\n",
        "\n",
        "### Likely Student Questions and Answers\n",
        "\n",
        "**Q: Why do we need to create a custom dataset class like `IMDBDataset`?**\n",
        "- **A: A custom dataset class allows us to preprocess and format the data specifically for our model, ensuring that each sample is correctly tokenized and ready for input into the BERT model.**\n",
        "\n",
        "**Q: How does `BertTokenizer` help in preparing the text data?**\n",
        "- **A: `BertTokenizer` converts raw text into token IDs that BERT can process, adds special tokens required by BERT, and creates attention masks, handling details that are crucial for the model to understand the input properly.**\n",
        "\n",
        "**Q: What happens if the text length exceeds `MAX_LEN` during tokenization?**\n",
        "- **A: If the text length exceeds `MAX_LEN`, it will be truncated to fit within the maximum length, ensuring that all inputs are of a consistent size.**\n",
        "\n",
        "**Q: Why is padding important in tokenization?**\n",
        "- **A: Padding ensures that all input sequences are of the same length, which is necessary for batch processing in neural networks. It also helps in efficiently utilizing computational resources by standardizing input dimensions.**\n",
        "\n",
        "**Q: How do the attention masks benefit the model?**\n",
        "- **A: Attention masks help the model distinguish between actual tokens and padding tokens, allowing it to focus on the meaningful parts of the input and ignore the padding.**\n",
        "\n",
        "**Q: What should we do if we encounter an error with tokenization?**\n",
        "- **A: Check the input text for any anomalies, ensure that the tokenizer is correctly loaded, and verify that the `MAX_LEN` parameter is appropriately set. Consulting the BERT tokenizer documentation for specific error messages can also help troubleshoot issues.**\n",
        "\n",
        "**Q: Can we use a different pretrained tokenizer with this dataset class?**\n",
        "- **A: Yes, you can use a different pretrained tokenizer by modifying the `BertTokenizer.from_pretrained` line to load the desired tokenizer, as long as it supports the same methods used in the class.**\n",
        "\n",
        "**Q: Why do we use `torch.tensor(label, dtype=torch.long)` for the labels?**\n",
        "- **A: Converting labels to long tensors ensures compatibility with PyTorch's loss functions, which typically expect target labels to be in this format for classification tasks.**\n",
        "\n",
        "**Q: What is the impact of truncation on the model's performance?**\n",
        "- **A: Truncation can lead to loss of information if important parts of the text are cut off. It's a trade-off between managing input size and preserving as much information as possible. Adjusting `MAX_LEN` based on the dataset can help mitigate this issue.**\n"
      ],
      "metadata": {
        "id": "iWej2ZZAazVg"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "class IMDBDataset(Dataset):\n",
        "    def __init__(self, texts, labels):\n",
        "        self.texts = texts\n",
        "        self.labels = labels\n",
        "        self.tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')\n",
        "\n",
        "    def __len__(self):\n",
        "        return len(self.texts)\n",
        "\n",
        "    def __getitem__(self, idx):\n",
        "        text = self.texts[idx]\n",
        "        label = self.labels[idx]\n",
        "        encoding = self.tokenizer.encode_plus(\n",
        "            text,\n",
        "            add_special_tokens=True,\n",
        "            max_length=MAX_LEN,\n",
        "            return_token_type_ids=False,\n",
        "            padding='max_length',\n",
        "            return_attention_mask=True,\n",
        "            truncation=True,\n",
        "            return_tensors='pt'\n",
        "        )\n",
        "        return {\n",
        "            'input_ids': encoding['input_ids'].flatten(),\n",
        "            'attention_mask': encoding['attention_mask'].flatten(),\n",
        "            'labels': torch.tensor(label, dtype=torch.long)\n",
        "        }"
      ],
      "metadata": {
        "id": "ydBw9rxV_v0l"
      },
      "execution_count": 16,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Explanation of the Dataset Loading and DataLoader Creation Code and Q&A\n",
        "\n",
        "1. **Q: What does `dataset = load_dataset(\"imdb\")` do?**\n",
        "   - **A: It loads the IMDB dataset using the `datasets` library, which includes train and test splits of movie reviews and their corresponding sentiment labels.**\n",
        "\n",
        "2. **Q: Why do we use `dataset['train'].shuffle(seed=42).select(range(int(len(dataset['train']) * 0.3)))`?**\n",
        "   - **A: This code shuffles the training dataset and selects 30% of it to create a smaller, manageable dataset for training. The `seed=42` ensures reproducibility.**\n",
        "\n",
        "3. **Q: What is the purpose of `train_test_split` in this context?**\n",
        "   - **A: `train_test_split` splits the reduced dataset into training and testing sets, with 80% of the data used for training and 20% for testing.**\n",
        "\n",
        "4. **Q: What does the `create_data_loader` function do?**\n",
        "   - **A: It creates a data loader for the given texts and labels by instantiating the `IMDBDataset` class and returning a `DataLoader` object with the specified batch size.**\n",
        "\n",
        "5. **Q: Why do we pass `num_workers=2` to the `DataLoader`?**\n",
        "   - **A: It sets the number of subprocesses to use for data loading, which can speed up data loading by parallelizing it.**\n",
        "\n",
        "6. **Q: How are the `train_data_loader` and `test_data_loader` created?**\n",
        "   - **A: By calling the `create_data_loader` function with the training and testing texts and labels, and the specified batch size.**\n",
        "\n",
        "### Likely Student Questions and Answers\n",
        "\n",
        "**Q: Why do we shuffle the training dataset before selecting a subset?**\n",
        "- **A: Shuffling ensures that the subset is representative of the entire dataset, reducing bias that could arise from the order of the data.**\n",
        "\n",
        "**Q: What is the significance of using `seed=42` in the shuffle method?**\n",
        "- **A: Setting a seed ensures that the shuffling is reproducible, meaning that the same subset will be selected each time the code is run.**\n",
        "\n",
        "**Q: Why do we only use 30% of the original training dataset?**\n",
        "- **A: Using a smaller subset of the dataset can make training faster and more manageable, especially when experimenting with model architectures or hyperparameters.**\n",
        "\n",
        "**Q: What does `test_size=0.2` mean in the `train_test_split` function?**\n",
        "- **A: It specifies that 20% of the reduced dataset will be used as the test set, while the remaining 80% will be used for training.**\n",
        "\n",
        "**Q: How does the `create_data_loader` function benefit the training process?**\n",
        "- **A: It encapsulates the creation of the dataset and data loader into a reusable function, ensuring consistent data loading and preprocessing.**\n",
        "\n",
        "**Q: Why do we need data loaders in PyTorch?**\n",
        "- **A: Data loaders handle the batching, shuffling, and loading of data in parallel, which is essential for efficient training and evaluation of models.**\n",
        "\n",
        "**Q: What happens if we increase the `batch_size` in the data loader?**\n",
        "- **A: Increasing the batch size means that more samples will be processed at once, which can speed up training but also requires more memory. There's a trade-off between training speed and memory usage.**\n",
        "\n",
        "**Q: Why is `num_workers` set to 2, and can we change it?**\n",
        "- **A: `num_workers=2` means that two subprocesses will be used for data loading, which can speed up the process. This number can be adjusted based on the available CPU resources and the specific dataset. More workers can further speed up data loading but also use more CPU resources.**\n",
        "\n",
        "**Q: How do we handle potential imbalances in the dataset after shuffling and splitting?**\n",
        "- **A: Ensuring that the split is stratified (maintaining the proportion of labels) can help. Techniques like oversampling or undersampling can also be used to address any imbalances.**\n",
        "\n",
        "**Q: Can we use a different dataset with the same `create_data_loader` function?**\n",
        "- **A: Yes, as long as the dataset is in a format compatible with the `IMDBDataset` class, the `create_data_loader` function can be used to create data loaders for any text classification dataset.**\n",
        "```"
      ],
      "metadata": {
        "id": "fkvEtFbwYpTP"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Load dataset\n",
        "dataset = load_dataset(\"imdb\")\n",
        "reduced_dataset = dataset['train'].shuffle(seed=42).select(range(int(len(dataset['train']) * 0.3)))\n",
        "\n",
        "# Split the reduced dataset into training and testing sets\n",
        "train_texts, test_texts, train_labels, test_labels = train_test_split(\n",
        "    reduced_dataset['text'], reduced_dataset['label'], test_size=0.2, random_state=42)\n",
        "\n",
        "# Data Loaders\n",
        "def create_data_loader(texts, labels, batch_size):\n",
        "    ds = IMDBDataset(texts, labels)\n",
        "    return DataLoader(ds, batch_size=batch_size, num_workers=2)\n",
        "\n",
        "# Create data loaders\n",
        "train_data_loader = create_data_loader(train_texts, train_labels, BATCH_SIZE)\n",
        "test_data_loader = create_data_loader(test_texts, test_labels, BATCH_SIZE)\n"
      ],
      "metadata": {
        "id": "VsfnBiv8V-Qn"
      },
      "execution_count": 17,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Setting up the models"
      ],
      "metadata": {
        "id": "r3vBFPuQDKVg"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "We will use: RNN (Recurrent Neural Network), LSTM (Long Short-Term Memory), and GRU (Gated Recurrent Units) to do sentiment analysis on the IMDB movie reviews dataset.\n",
        "\n",
        "### Explanation of the TextClassifier Class and Q&A\n",
        "\n",
        "1. **Q: What is the purpose of the `TextClassifier` class?**\n",
        "   - **A: The `TextClassifier` class is a neural network model for text classification, supporting RNN, LSTM, and GRU architectures.**\n",
        "\n",
        "2. **Q: What does the `__init__` method do?**\n",
        "   - **A: It initializes the model, including an embedding layer, a recurrent layer (RNN, LSTM, or GRU based on the `model_type`), and a fully connected layer.**\n",
        "\n",
        "3. **Q: Why is `nn.Embedding(30522, 256)` used?**\n",
        "   - **A: It creates an embedding layer with a vocabulary size of 30522 (BERT's vocab size) and an embedding dimension of 256, converting token IDs into dense vectors.**\n",
        "\n",
        "4. **Q: How does the `model_type` parameter affect the model?**\n",
        "   - **A: It determines which type of recurrent layer (RNN, LSTM, or GRU) is used in the model.**\n",
        "\n",
        "5. **Q: What are the dimensions of the input and hidden states in the recurrent layers?**\n",
        "   - **A: The input dimension is 256 (embedding size), and the hidden state dimension is 128.**\n",
        "\n",
        "6. **Q: What does the `forward` method do?**\n",
        "   - **A: It defines the forward pass of the model, embedding the input IDs, passing them through the recurrent layer, and then through a fully connected layer to produce the output.**\n",
        "\n",
        "7. **Q: Why is `batch_first=True` used in the recurrent layers?**\n",
        "   - **A: It indicates that the input and output tensors are expected to have the batch size as the first dimension, making the data format (batch_size, sequence_length, features).**\n",
        "\n",
        "8. **Q: How is the output of the recurrent layer processed in the forward method?**\n",
        "   - **A: The output of the recurrent layer is processed by taking the hidden state (for LSTM, the hidden state from the tuple), squeezing it to remove unnecessary dimensions, and passing it through the fully connected layer.**\n",
        "\n",
        "### Likely Student Questions and Answers\n",
        "\n",
        "**Q: What is the role of the embedding layer in the model?**\n",
        "- **A: The embedding layer converts input token IDs into dense vectors of a specified size (256 in this case), capturing semantic information about the tokens.**\n",
        "\n",
        "**Q: How does the model handle different types of recurrent layers?**\n",
        "- **A: The model uses an `if-elif` structure to initialize either an RNN, LSTM, or GRU layer based on the `model_type` parameter passed during initialization.**\n",
        "\n",
        "**Q: What happens if an unsupported `model_type` is passed to the `TextClassifier`?**\n",
        "- **A: The current implementation does not handle unsupported `model_type` values explicitly, which could lead to an error. It's good practice to add error handling for invalid `model_type` values.**\n",
        "\n",
        "**Q: Why do we check if `hidden` is a tuple in the forward method?**\n",
        "- **A: LSTM layers return a tuple (hidden state, cell state), while RNN and GRU return just the hidden state. The check ensures the correct hidden state is used for further processing.**\n",
        "\n",
        "**Q: What is the significance of `hidden.squeeze(0)` in the forward method?**\n",
        "- **A: `squeeze(0)` removes the first dimension if it is 1, effectively reshaping the hidden state tensor to the correct size for input to the fully connected layer.**\n",
        "\n",
        "**Q: How does the model handle variable-length sequences?**\n",
        "- **A: The current implementation assumes fixed-length sequences based on padding. For variable-length sequences, additional handling (like packing padded sequences) would be needed.**\n",
        "\n",
        "**Q: Why do we need a fully connected layer at the end of the model?**\n",
        "- **A: The fully connected layer maps the final hidden state to the output space, producing logits for the classification task (2 classes in this case).**\n",
        "\n",
        "**Q: Can this model be used for multi-class classification tasks?**\n",
        "- **A: Yes, the model can be adapted for multi-class classification by changing the output dimension of the fully connected layer to the number of classes.**\n",
        "\n",
        "**Q: How does the choice of RNN, LSTM, or GRU affect the model's performance?**\n",
        "- **A: RNNs are simpler but can struggle with long-term dependencies. LSTMs and GRUs are more complex and better at capturing long-term dependencies, often leading to better performance on sequential data tasks.**\n",
        "\n",
        "**Q: What adjustments are needed if we want to use a different pre-trained tokenizer?**\n",
        "- **A: Ensure the vocabulary size in the embedding layer matches the new tokenizer's vocabulary size. Also, make sure the tokenization and preprocessing steps are compatible with the new tokenizer.**\n",
        "```\n",
        "\n",
        "Recurrent Neural Networks (RNNs), Long Short-Term Memory (LSTM) networks, and Gated Recurrent Units (GRUs) are all types of recurrent neural networks designed to handle sequential data. Here are the key differences between them:\n",
        "\n",
        "### Recurrent Neural Networks (RNNs)\n",
        "\n",
        "- **Structure**: RNNs have a simple structure where the output from the previous step is fed as input to the current step. This creates a loop that allows information to persist.\n",
        "- **Vanishing Gradient Problem**: RNNs often suffer from the vanishing gradient problem, making it difficult to learn long-term dependencies. This happens because the gradients used during backpropagation can become very small, effectively stopping the learning process for earlier layers in long sequences.\n",
        "- **Use Case**: Suitable for simple sequence data where long-term dependencies are not critical.\n",
        "\n",
        "### Long Short-Term Memory (LSTM) Networks\n",
        "\n",
        "- **Structure**: LSTMs have a more complex structure, including a cell state and three gates (input gate, forget gate, and output gate). These gates regulate the flow of information into and out of the cell state, allowing the network to maintain and update long-term dependencies.\n",
        "- **Forget Gate**: Decides which information to discard from the cell state.\n",
        "- **Input Gate**: Decides which information to update in the cell state.\n",
        "- **Output Gate**: Decides the output based on the cell state and the input.\n",
        "- **Ability to Handle Long-Term Dependencies**: LSTMs are designed to remember information for long periods, effectively addressing the vanishing gradient problem.\n",
        "- **Use Case**: Suitable for tasks that require learning and maintaining long-term dependencies, such as language modeling, translation, and time series prediction.\n",
        "\n",
        "### Gated Recurrent Units (GRUs)\n",
        "\n",
        "- **Structure**: GRUs are similar to LSTMs but have a simpler architecture. They consist of two gates: a reset gate and an update gate.\n",
        "- **Reset Gate**: Determines how much of the past information to forget.\n",
        "- **Update Gate**: Decides how much of the past information to carry forward to the future.\n",
        "- **Fewer Parameters**: GRUs have fewer parameters compared to LSTMs, making them computationally more efficient while still being able to handle long-term dependencies.\n",
        "- **Ability to Handle Long-Term Dependencies**: GRUs, like LSTMs, address the vanishing gradient problem and can handle long-term dependencies, though with a simpler mechanism.\n",
        "- **Use Case**: Suitable for tasks that require maintaining long-term dependencies with a preference for computational efficiency, such as real-time processing tasks and cases where a faster model is needed.\n",
        "\n",
        "### Summary\n",
        "\n",
        "- **RNNs**: Simple structure, suffers from vanishing gradient problem, suitable for short-term dependencies.\n",
        "- **LSTMs**: Complex structure with three gates, handles long-term dependencies well, computationally intensive.\n",
        "- **GRUs**: Simplified version of LSTMs with two gates, handles long-term dependencies efficiently, fewer parameters and faster training compared to LSTMs.\n",
        "\n",
        "Each type of network has its strengths and is chosen based on the specific requirements of the task at hand."
      ],
      "metadata": {
        "id": "XxnnlW9Va-D3"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "This code defines a neural network model for text classification using different types of recurrent neural networks (RNN, LSTM, GRU). The `TextClassifier` class is initialized with a specific model type ('rnn', 'lstm', or 'gru'), which determines the type of RNN used. An embedding layer converts input tokens into dense vectors, which are then processed by the chosen RNN to capture sequential information. The final hidden state from the RNN is passed through a fully connected layer to produce the output, which in this case is a binary classification. The `forward` method defines the forward pass of the model, handling the input data and returning the classification result."
      ],
      "metadata": {
        "id": "sifHKMI5aBxq"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Neural Network Model with RNN, LSTM, and GRU\n",
        "class TextClassifier(nn.Module):\n",
        "    def __init__(self, model_type):\n",
        "        super(TextClassifier, self).__init__()\n",
        "        self.embedding = nn.Embedding(30522, 256)  # Bert vocab size is 30522\n",
        "        if model_type == 'rnn':\n",
        "            self.rnn = nn.RNN(256, 128, batch_first=True)\n",
        "        elif model_type == 'lstm':\n",
        "            self.rnn = nn.LSTM(256, 128, batch_first=True)\n",
        "        elif model_type == 'gru':\n",
        "            self.rnn = nn.GRU(256, 128, batch_first=True)\n",
        "        self.fc = nn.Linear(128, 2)\n",
        "\n",
        "    def forward(self, input_ids, attention_mask):\n",
        "        x = self.embedding(input_ids)\n",
        "        output, hidden = self.rnn(x)\n",
        "        if isinstance(hidden, tuple):  # LSTM returns (hidden, cell)\n",
        "            hidden = hidden[0]\n",
        "        hidden = hidden.squeeze(0)\n",
        "        out = self.fc(hidden)\n",
        "        return out"
      ],
      "metadata": {
        "id": "FtOfv96lO7p3"
      },
      "execution_count": 18,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "We could play with the following aspects of the model setup:\n",
        "\n",
        "`embedding` defines the embedding layer, with 2 parameters: (1) the size of the input dimension (i.e, the BERT vocabulary size). This means the layer can represent each of the 30,522 unique tokens. Generally speaking, a larger number of dimensions can capture more complex features, with costs of computational resources and risks of overfitting; and (2) size of each embedding vector. In our current setup, each token from the vocabulary mentioned above is mapped to a 256-dimensional vector that represents learned features of that token.\n",
        "\n"
      ],
      "metadata": {
        "id": "13Py78bey_2A"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "RNN/LSTM/GRU Layers:\n",
        "\n",
        "Parameters of these layers are: (1), number of expected features in the input; (2), the number of features that the model is expecte to capture. This parameter determines the model's ability to learn  complex patterns. (3), batch_first. This parameter affects the data structure or format that we give to layers.  RNN, LSTM, and GRU layers expects `atch_first=False`, but other layers expects True. batch_first=True gives more intuitive data format and makes data format compatible when integrating RNN, LSTM, and GRU outputs with other components like fully connected layers.\n"
      ],
      "metadata": {
        "id": "RsIyBOEQ0De3"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Model training and evaluation\n",
        "\n",
        "### Explanation of the Training and Evaluation Functions and Q&A\n",
        "\n",
        "1. **Q: What is the purpose of the `train_epoch` function?**\n",
        "   - **A: The `train_epoch` function trains the model for one epoch, updating the model's parameters and calculating the accuracy and average loss for the training data.**\n",
        "\n",
        "2. **Q: Why do we use `model.train()` at the beginning of the `train_epoch` function?**\n",
        "   - **A: It sets the model to training mode, enabling behaviors like dropout and batch normalization which are only active during training.**\n",
        "\n",
        "3. **Q: What does `input_ids = d['input_ids'].to(device)` do?**\n",
        "   - **A: It moves the input IDs to the specified device (GPU or CPU) for computation.**\n",
        "\n",
        "4. **Q: How is the loss calculated in the `train_epoch` function?**\n",
        "   - **A: The loss is calculated using the provided loss function (`loss_fn`) by comparing the model's outputs with the true labels.**\n",
        "\n",
        "5. **Q: Why do we call `optimizer.zero_grad()` before `loss.backward()`?**\n",
        "   - **A: To clear any accumulated gradients from previous iterations, ensuring that the gradients are correctly calculated for the current batch.**\n",
        "\n",
        "6. **Q: What is the purpose of `loss.backward()` and `optimizer.step()`?**\n",
        "   - **A: `loss.backward()` computes the gradients of the loss with respect to the model's parameters, and `optimizer.step()` updates the parameters based on these gradients.**\n",
        "\n",
        "7. **Q: How is accuracy calculated in the `train_epoch` function?**\n",
        "   - **A: Accuracy is calculated by summing the correct predictions and dividing by the total number of samples in the dataset.**\n",
        "\n",
        "8. **Q: What does the `eval_model` function do?**\n",
        "   - **A: The `eval_model` function evaluates the model's performance on the validation or test data, returning the accuracy and average loss without updating the model's parameters.**\n",
        "\n",
        "9. **Q: Why do we use `model.eval()` at the beginning of the `eval_model` function?**\n",
        "   - **A: It sets the model to evaluation mode, disabling behaviors like dropout and batch normalization that are only used during training.**\n",
        "\n",
        "10. **Q: Why is `torch.no_grad()` used in the `eval_model` function?**\n",
        "    - **A: It disables gradient calculation, reducing memory usage and speeding up computations during evaluation.**\n",
        "\n",
        "11. **Q: How is the average loss calculated in both functions?**\n",
        "    - **A: The average loss is calculated by taking the mean of the losses recorded for each batch.**\n",
        "\n",
        "### Likely Student Questions and Answers\n",
        "\n",
        "**Q: Why do we need separate functions for training and evaluation?**\n",
        "- **A: Training and evaluation require different behaviors. Training involves updating the model's parameters using backpropagation, while evaluation only assesses the model's performance on validation or test data without updating the parameters.**\n",
        "\n",
        "**Q: What is the role of the `optimizer` in the `train_epoch` function?**\n",
        "- **A: The optimizer updates the model's parameters based on the gradients calculated during backpropagation, helping the model to minimize the loss function.**\n",
        "\n",
        "**Q: How do we ensure that the model's parameters are not updated during evaluation?**\n",
        "- **A: By setting the model to evaluation mode with `model.eval()` and using `torch.no_grad()` to disable gradient calculations.**\n",
        "\n",
        "**Q: Why do we calculate the mean of the losses?**\n",
        "- **A: Calculating the mean of the losses provides a single value that summarizes the model's performance across all batches, making it easier to monitor training progress.**\n",
        "\n",
        "**Q: What does `correct_predictions.double() / len(data_loader.dataset)` compute?**\n",
        "- **A: It computes the accuracy of the model by dividing the number of correct predictions by the total number of samples in the dataset.**\n",
        "\n",
        "**Q: Can we modify these functions to support other types of models?**\n",
        "- **A: Yes, these functions can be adapted to other types of models by ensuring the forward pass and loss calculation are compatible with the new model.**\n",
        "\n",
        "**Q: What happens if `optimizer.zero_grad()` is not called?**\n",
        "- **A: Gradients would accumulate from previous batches, leading to incorrect gradient calculations and potentially destabilizing the training process.**\n",
        "\n",
        "**Q: Why is it important to set the model to evaluation mode during evaluation?**\n",
        "- **A: Setting the model to evaluation mode ensures that layers like dropout and batch normalization behave correctly, providing accurate assessment of the model's performance.**\n",
        "\n",
        "**Q: How can we track the training progress over multiple epochs?**\n",
        "- **A: By storing the accuracy and loss values returned by `train_epoch` and `eval_model` for each epoch, and then plotting or printing these values to monitor the training progress.**\n",
        "\n",
        "**Q: What if the model's performance is worse on the validation set than on the training set?**\n",
        "- **A: This could indicate overfitting, where the model performs well on training data but poorly on unseen data. Regularization techniques, more data, or a simpler model might help address this issue.**\n",
        "```"
      ],
      "metadata": {
        "id": "nrixsDQlDQN8"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Training function\n",
        "def train_epoch(model, data_loader, loss_fn, optimizer, device):\n",
        "    model.train()\n",
        "    losses = []\n",
        "    correct_predictions = 0\n",
        "    for d in data_loader:\n",
        "        input_ids = d['input_ids'].to(device)\n",
        "        attention_mask = d['attention_mask'].to(device)\n",
        "        labels = d['labels'].to(device)\n",
        "        outputs = model(input_ids, attention_mask)\n",
        "        _, preds = torch.max(outputs, dim=1)\n",
        "        loss = loss_fn(outputs, labels)\n",
        "        correct_predictions += torch.sum(preds == labels)\n",
        "        losses.append(loss.item())\n",
        "        optimizer.zero_grad()\n",
        "        loss.backward()\n",
        "        optimizer.step()\n",
        "    return correct_predictions.double() / len(data_loader.dataset), np.mean(losses)\n",
        "\n",
        "# Evaluate function\n",
        "def eval_model(model, data_loader, loss_fn, device):\n",
        "    model.eval()\n",
        "    losses = []\n",
        "    correct_predictions = 0\n",
        "    with torch.no_grad():\n",
        "        for d in data_loader:\n",
        "            input_ids = d['input_ids'].to(device)\n",
        "            attention_mask = d['attention_mask'].to(device)\n",
        "            labels = d['labels'].to(device)\n",
        "            outputs = model(input_ids, attention_mask)\n",
        "            _, preds = torch.max(outputs, dim=1)\n",
        "            loss = loss_fn(outputs, labels)\n",
        "            correct_predictions += torch.sum(preds == labels)\n",
        "            losses.append(loss.item())\n",
        "    return correct_predictions.double() / len(data_loader.dataset), np.mean(losses)"
      ],
      "metadata": {
        "id": "f4BAHNwFDP1W"
      },
      "execution_count": 19,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now, we train the models and get a look at the accuracy along with other evaluation metrics on test data."
      ],
      "metadata": {
        "id": "UYPyDyrySt1D"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "The following code should take ~7 min to run\n",
        "\n",
        "### Explanation of the Training and Evaluation Loop and Q&A\n",
        "\n",
        "1. **Q: What is the purpose of the `model_types` list?**\n",
        "   - **A: It contains the different types of models (`rnn`, `lstm`, `gru`) that will be trained and evaluated.**\n",
        "\n",
        "2. **Q: Why do we use a loop to iterate over `model_types`?**\n",
        "   - **A: To train and evaluate each model type (`rnn`, `lstm`, `gru`) sequentially, allowing comparison of their performance.**\n",
        "\n",
        "3. **Q: What does `model = TextClassifier(model_type).to(DEVICE)` do?**\n",
        "   - **A: It initializes a `TextClassifier` model of the specified type and moves it to the specified device (GPU or CPU).**\n",
        "\n",
        "4. **Q: What is the role of `nn.CrossEntropyLoss().to(DEVICE)` in the code?**\n",
        "   - **A: It defines the loss function used for training, and moves it to the specified device.**\n",
        "\n",
        "5. **Q: How is the optimizer defined for the model?**\n",
        "   - **A: The optimizer is defined using `torch.optim.Adam` with the model's parameters and a specified learning rate.**\n",
        "\n",
        "6. **Q: What happens inside the training loop for each epoch?**\n",
        "   - **A: The model is trained for one epoch using the `train_epoch` function, and the training loss and accuracy are printed.**\n",
        "\n",
        "7. **Q: Why do we set `model.eval()` before evaluation?**\n",
        "   - **A: To set the model to evaluation mode, disabling behaviors like dropout and batch normalization that are only used during training.**\n",
        "\n",
        "8. **Q: How are the final outputs and targets collected for evaluation?**\n",
        "   - **A: They are collected by iterating over the test data loader, making predictions, and storing the predicted and true labels.**\n",
        "\n",
        "9. **Q: What metrics are calculated for model evaluation?**\n",
        "   - **A: Accuracy, precision, recall, F1-score, and loss are calculated to evaluate the model's performance.**\n",
        "\n",
        "10. **Q: Why do we use `torch.no_grad()` during evaluation?**\n",
        "    - **A: To disable gradient calculation, reducing memory usage and speeding up computations during evaluation.**\n",
        "\n",
        "11. **Q: How are the evaluation results stored and printed?**\n",
        "    - **A: The results for each model type are stored in a dictionary and printed after evaluation.\n",
        "\n",
        "### Likely Student Questions and Answers\n",
        "\n",
        "**Q: Why do we train and evaluate multiple types of models (RNN, LSTM, GRU)?**\n",
        "- **A: Training and evaluating multiple model types allows us to compare their performance and choose the best one for the specific task.**\n",
        "\n",
        "**Q: What is the significance of moving the model and loss function to the specified device?**\n",
        "- **A: Moving the model and loss function to the device ensures that computations are performed on the GPU if available, which speeds up training and evaluation.**\n",
        "\n",
        "**Q: Why do we use `torch.optim.Adam` as the optimizer?**\n",
        "- **A: Adam is a popular optimization algorithm that adjusts the learning rate dynamically based on the gradients, often leading to faster convergence compared to simpler optimizers like SGD.**\n",
        "\n",
        "**Q: How is the training accuracy and loss calculated during each epoch?**\n",
        "- **A: The training accuracy and loss are calculated by the `train_epoch` function, which processes the entire training dataset and returns the average loss and accuracy.**\n",
        "\n",
        "**Q: Why is it important to evaluate the model on a separate test set?**\n",
        "- **A: Evaluating on a separate test set provides an unbiased estimate of the model's performance on unseen data, helping to detect overfitting and assess generalization.**\n",
        "\n",
        "**Q: How do we ensure the evaluation is performed correctly without updating the model's parameters?**\n",
        "- **A: By setting the model to evaluation mode with `model.eval()` and using `torch.no_grad()` to disable gradient computations during evaluation.**\n",
        "\n",
        "**Q: What is the purpose of calculating precision, recall, and F1-score in addition to accuracy?**\n",
        "- **A: Precision, recall, and F1-score provide more detailed insights into the model's performance, especially in cases of imbalanced classes where accuracy alone may be misleading.**\n",
        "\n",
        "**Q: How do we calculate the average loss during evaluation?**\n",
        "- **A: The average loss is calculated by taking the mean of the losses for each batch in the test data loader.**\n",
        "\n",
        "**Q: Why do we print the results for each model type after evaluation?**\n",
        "- **A: Printing the results allows us to compare the performance of different model types and determine which one performs best for the task.**\n",
        "\n",
        "**Q: What should we do if the model's performance is not satisfactory?**\n",
        "- **A: Consider tuning hyperparameters (learning rate, batch size, etc.), using a different model architecture, increasing the training data, or improving data preprocessing.**\n",
        "\n",
        "**Q: How can we handle the scenario where the dataset is imbalanced?**\n",
        "- **A: Techniques like oversampling minority classes, undersampling majority classes, using weighted loss functions, or data augmentation can help address class imbalance.**\n",
        "\n",
        "\n",
        "{'rnn': (0.5053333333333333, 0.7139620933126896), 'lstm': (0.6433333333333333, 0.809315027391657), 'gru': (0.722, 0.6452926746074189)}\n",
        "\n",
        "\n",
        "The basic RNN model has the lowest accuracy among the three, and has a relatively high loss. The model might be underfitting.\n",
        "\n",
        "LSTM models perform better than basic RNNs, as they can keep track of important information throughout the movie reviews. This is particularly important for long reviews, as the information at the beginning of the review might make sense by considering the full review together. LSTM that retains memory when analyzing such data has a higher accuracy.\n",
        "\n",
        "GRUs is best among the three. Similar to LSTM, GRU can account for memory and can help avoid overfit."
      ],
      "metadata": {
        "id": "oRsSIooV4K3Y"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from torch.utils.data import DataLoader\n",
        "import numpy as np\n",
        "import torch\n",
        "import torch.nn as nn\n",
        "from tabulate import tabulate\n",
        "from sklearn.metrics import accuracy_score, precision_recall_fscore_support\n",
        "\n",
        "model_types = ['rnn', 'lstm', 'gru']\n",
        "results = {}\n",
        "for model_type in model_types:\n",
        "    print(f\"Training and evaluating model: {model_type.upper()}\")\n",
        "    model = TextClassifier(model_type).to(DEVICE)\n",
        "    loss_fn = nn.CrossEntropyLoss().to(DEVICE)\n",
        "    optimizer = torch.optim.Adam(model.parameters(), lr=LEARNING_RATE)\n",
        "\n",
        "    for epoch in range(EPOCHS):\n",
        "        print(f'Epoch {epoch + 1}/{EPOCHS}')\n",
        "        train_acc, train_loss = train_epoch(\n",
        "            model, train_data_loader, loss_fn, optimizer, DEVICE)\n",
        "        print(f'Train loss {train_loss} accuracy {train_acc}')\n",
        "\n",
        "    # Evaluation\n",
        "    model.eval()\n",
        "    final_outputs = []\n",
        "    final_targets = []\n",
        "    with torch.no_grad():\n",
        "        for d in test_data_loader:\n",
        "            input_ids = d['input_ids'].to(DEVICE)\n",
        "            attention_mask = d['attention_mask'].to(DEVICE)\n",
        "            labels = d['labels'].to(DEVICE)\n",
        "            outputs = model(input_ids, attention_mask)\n",
        "            _, preds = torch.max(outputs, dim=1)\n",
        "            final_outputs.extend(preds.cpu().numpy())\n",
        "            final_targets.extend(labels.cpu().numpy())\n",
        "\n",
        "    test_acc = accuracy_score(final_targets, final_outputs)\n",
        "    precision, recall, f1, _ = precision_recall_fscore_support(final_targets, final_outputs, average='binary')\n",
        "    test_loss = np.mean([\n",
        "        loss_fn(\n",
        "            model(d['input_ids'].clone().detach().to(DEVICE), d['attention_mask'].clone().detach().to(DEVICE)),\n",
        "            d['labels'].clone().detach().to(DEVICE)  # Ensure labels are on the same device as the model\n",
        "        ).item()\n",
        "        for d in test_data_loader\n",
        "    ])\n",
        "\n",
        "\n",
        "    results[model_type] = {\n",
        "        'accuracy': test_acc,\n",
        "        'precision': precision,\n",
        "        'recall': recall,\n",
        "        'f1_score': f1,\n",
        "        'loss': test_loss\n",
        "    }\n",
        "    print(f\"Results for {model_type.upper()}: Accuracy: {test_acc}, Precision: {precision}, Recall: {recall}, F1-score: {f1}, Loss: {test_loss}\")\n",
        "\n",
        "\n",
        "def format_results(results):\n",
        "    # Prepare the data for tabulation\n",
        "    data = []\n",
        "    for model_type, metrics in results.items():\n",
        "        row = [model_type.upper()] + [metrics[m] for m in ['accuracy', 'precision', 'recall', 'f1_score', 'loss']]\n",
        "        data.append(row)\n",
        "\n",
        "    # Define headers\n",
        "    headers = [\"Model\", \"Accuracy\", \"Precision\", \"Recall\", \"F1-score\", \"Loss\"]\n",
        "\n",
        "    # Create the table\n",
        "    table = tabulate(data, headers=headers, tablefmt=\"grid\", floatfmt=\".4f\")\n",
        "\n",
        "    return table\n",
        "\n",
        "# Assuming 'results' is the dictionary returned by your code\n",
        "print(\"RNN Models Comparison Results\")\n",
        "print(format_results(results))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VLqzKXtM5lHr",
        "outputId": "2c2964e8-c670-4cbb-d493-b70b179631b8"
      },
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Training and evaluating model: RNN\n",
            "Epoch 1/3\n",
            "Train loss 0.7048776529630025 accuracy 0.5071666666666667\n",
            "Epoch 2/3\n",
            "Train loss 0.6862995425860087 accuracy 0.5365\n",
            "Epoch 3/3\n",
            "Train loss 0.6419523843924204 accuracy 0.5875\n",
            "Results for RNN: Accuracy: 0.5026666666666667, Precision: 0.5187165775401069, Recall: 0.25526315789473686, F1-score: 0.3421516754850088, Loss: 0.7325109674575481\n",
            "Training and evaluating model: LSTM\n",
            "Epoch 1/3\n",
            "Train loss 0.6961345068613688 accuracy 0.5151666666666667\n",
            "Epoch 2/3\n",
            "Train loss 0.6680211482048035 accuracy 0.5678333333333333\n",
            "Epoch 3/3\n",
            "Train loss 0.565173053264618 accuracy 0.6425\n",
            "Results for LSTM: Accuracy: 0.5413333333333333, Precision: 0.6104294478527608, Recall: 0.2618421052631579, F1-score: 0.3664825046040516, Loss: 0.8849392431847592\n",
            "Training and evaluating model: GRU\n",
            "Epoch 1/3\n",
            "Train loss 0.7018924129803975 accuracy 0.528\n",
            "Epoch 2/3\n",
            "Train loss 0.6047081172466278 accuracy 0.6686666666666666\n",
            "Epoch 3/3\n",
            "Train loss 0.39581112549702324 accuracy 0.8256666666666667\n",
            "Results for GRU: Accuracy: 0.6846666666666666, Precision: 0.6773794808405439, Recall: 0.7210526315789474, F1-score: 0.6985340981516891, Loss: 0.7028203397355182\n",
            "RNN Models Comparison Results\n",
            "+---------+------------+-------------+----------+------------+--------+\n",
            "| Model   |   Accuracy |   Precision |   Recall |   F1-score |   Loss |\n",
            "+=========+============+=============+==========+============+========+\n",
            "| RNN     |     0.5027 |      0.5187 |   0.2553 |     0.3422 | 0.7325 |\n",
            "+---------+------------+-------------+----------+------------+--------+\n",
            "| LSTM    |     0.5413 |      0.6104 |   0.2618 |     0.3665 | 0.8849 |\n",
            "+---------+------------+-------------+----------+------------+--------+\n",
            "| GRU     |     0.6847 |      0.6774 |   0.7211 |     0.6985 | 0.7028 |\n",
            "+---------+------------+-------------+----------+------------+--------+\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Model evaluation visualization"
      ],
      "metadata": {
        "id": "H5uhSNak2Njl"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "### Explanation of the Code for Plotting Model Evaluation Metrics and Q&A\n",
        "\n",
        "1. **Q: What does `data = pd.DataFrame(results).T` do?**\n",
        "   - **A: It creates a DataFrame from the `results` dictionary, transposing it so that model types become the row indices and metrics become the columns.**\n",
        "\n",
        "2. **Q: Why do we use `data.reset_index(inplace=True)` after setting the index name?**\n",
        "   - **A: To reset the index so that the model types become a column, making it easier to plot the data.**\n",
        "\n",
        "3. **Q: What is the purpose of `fig, ax = plt.subplots(figsize=(10, 6))`?**\n",
        "   - **A: It creates a figure and a set of subplots with a specified figure size of 10 inches by 6 inches.**\n",
        "\n",
        "4. **Q: What does the `width = 0.15` variable specify?**\n",
        "   - **A: It sets the width of the bars in the bar plot.**\n",
        "\n",
        "5. **Q: How do we determine the metrics to be plotted?**\n",
        "   - **A: The metrics are determined by `metrics = data.columns[1:]`, which selects all columns except the first one (which contains the model types).**\n",
        "\n",
        "6. **Q: What does `x = np.arange(len(data['Model']))` do?**\n",
        "   - **A: It creates an array of indices corresponding to the number of model types, which is used for positioning the bars on the x-axis.**\n",
        "\n",
        "7. **Q: How are the bars for each metric added to the plot?**\n",
        "   - **A: A loop iterates over the metrics, and `ax.bar` adds bars to the plot for each metric, offsetting them on the x-axis to avoid overlap.**\n",
        "\n",
        "8. **Q: What does `ax.set_xticks(x + width * (len(metrics) - 1) / 2)` accomplish?**\n",
        "   - **A: It sets the x-ticks to be centered under the grouped bars by adjusting their positions.**\n",
        "\n",
        "9. **Q: How do we label the x-ticks with the model types?**\n",
        "   - **A: Using `ax.set_xticklabels(data['Model'])`, which sets the labels of the x-ticks to the model types.**\n",
        "\n",
        "10. **Q: What is the purpose of `fig.tight_layout()`?**\n",
        "    - **A: To automatically adjust the subplot parameters to give specified padding, ensuring that the subplots fit within the figure area without overlapping.**\n",
        "\n",
        "### Likely Student Questions and Answers\n",
        "\n",
        "**Q: Why do we transpose the `results` DataFrame initially?**\n",
        "- **A: Transposing the DataFrame makes the model types the row indices and metrics the columns, which is a more convenient format for plotting grouped bar charts.**\n",
        "\n",
        "**Q: How does resetting the index help in plotting?**\n",
        "- **A: Resetting the index moves the model types from the index to a column, making it easier to reference them when plotting.**\n",
        "\n",
        "**Q: Why do we set the width of the bars to 0.15?**\n",
        "- **A: The width of 0.15 is chosen to ensure that the bars for different metrics are clearly separated and do not overlap.**\n",
        "\n",
        "**Q: What does `np.arange(len(data['Model']))` achieve in the plot?**\n",
        "- **A: It generates an array of indices that correspond to the model types, providing positions for the bars on the x-axis.**\n",
        "\n",
        "**Q: Why is it important to adjust the x-tick positions when plotting grouped bars?**\n",
        "- **A: Adjusting the x-tick positions ensures that the ticks are centered under the grouped bars, making the plot easier to read and interpret.**\n",
        "\n",
        "**Q: How do we ensure that the subplots fit well within the figure area?**\n",
        "- **A: Using `fig.tight_layout()` adjusts the subplot parameters to avoid overlap and ensure that all elements fit neatly within the figure area.**\n",
        "\n",
        "**Q: What should we do if the plot is too cluttered or difficult to read?**\n",
        "- **A: Consider increasing the figure size, adjusting the bar width, or plotting fewer metrics at a time to improve readability.**\n",
        "\n",
        "**Q: Can we use this plotting code for other datasets?**\n",
        "- **A: Yes, as long as the data is in a similar format (with model types and corresponding metrics), this code can be adapted to plot other datasets.**\n",
        "\n",
        "**Q: Why do we use a loop to add bars for each metric?**\n",
        "- **A: Using a loop allows us to systematically add bars for each metric, ensuring that all metrics are included in the plot and are correctly positioned.**\n",
        "\n",
        "**Q: How does the `ax.legend()` function enhance the plot?**\n",
        "- **A: The legend helps identify which color corresponds to which metric, making it easier to interpret the grouped bar chart.**\n"
      ],
      "metadata": {
        "id": "qXjV7V1Qale1"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import pandas as pd\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "data = pd.DataFrame(results).T\n",
        "data.index.name = 'Model'\n",
        "data.reset_index(inplace=True)\n",
        "\n",
        "fig, ax = plt.subplots(figsize=(10, 6))\n",
        "width = 0.15\n",
        "\n",
        "metrics = data.columns[1:]\n",
        "x = np.arange(len(data['Model']))\n",
        "for i, metric in enumerate(metrics):\n",
        "    ax.bar(x + i * width, data[metric], width, label=metric)\n",
        "\n",
        "ax.set_xlabel('Model Type')\n",
        "ax.set_title('Model Evaluation Metrics by Model Type')\n",
        "ax.set_xticks(x + width * (len(metrics) - 1) / 2)\n",
        "ax.set_xticklabels(data['Model'])\n",
        "ax.legend()\n",
        "\n",
        "fig.tight_layout()\n",
        "plt.show()\n"
      ],
      "metadata": {
        "id": "PCX4KriaRrqz",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 607
        },
        "outputId": "53327a61-6fa1-4bc7-92d8-a850f7f7afd9"
      },
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 1000x600 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAA90AAAJOCAYAAACqS2TfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABVf0lEQVR4nO3de3zO9f/H8ee184ENbTbTmMNochgTOUWoOSRKETKEUvaTRg7lTDmfiS/lUBkqOnwjqrGSQ45DX8thmSlncho2ts/vD7dddbWNbXx2mR732+26tev9eX8+n9fnuq5dee79+bw/FsMwDAEAAAAAgLvOwd4FAAAAAABwvyJ0AwAAAABgEkI3AAAAAAAmIXQDAAAAAGASQjcAAAAAACYhdAMAAAAAYBJCNwAAAAAAJiF0AwAAAABgEkI3AAAAAAAmIXQDwH3MYrFoxIgRuV4vMTFRFotFixYtuus13S1BQUHq2rWrXfZdEF6f/Na1a1cFBQXl2/4sFosiIyPzbX/57U4+33n9vQcAmIPQDQAmW7RokSwWiywWi3766adMyw3DUGBgoCwWi5566ik7VJh3sbGx1mPL6rFs2TJ7l3hHoqOjNW3aNHuXYaNr166yWCzy8vLS1atXMy0/ePCg9fWfNGlSrrd/5coVjRgxQrGxsXeh2oLt75/vjz/+OMs+9erVk8ViUeXKlfO5urxr1KjRLX9vMx4EdwC4O5zsXQAA/Fu4ubkpOjpa9evXt2n/4Ycf9Pvvv8vV1dVOld25Pn366JFHHsnUXqdOHTtUc/dER0frl19+Ud++fW3aS5curatXr8rZ2dkudTk5OenKlSv673//q3bt2tksW7Jkidzc3HTt2rU8bfvKlSsaOXKkpJvhLKfmz5+v9PT0PO3zXpfxu/viiy/atCcmJmrTpk1yc3OzU2V58/bbb6tHjx7W59u2bdOMGTP01ltvKSQkxNpetWpVe5QHAPcdQjcA5JMWLVro008/1YwZM+Tk9NfXb3R0tMLCwnTmzBk7VndnGjRooOeee87eZeQbi8Vi16Dl6uqqevXqaenSpZlCd3R0tFq2bKkVK1bkSy3Jycny9PS02x8g8kOLFi301Vdf6cyZM/Lx8bG2R0dHy8/PT8HBwfrzzz/tWGHuPPHEEzbP3dzcNGPGDD3xxBO5+kMLACBnOL0cAPJJhw4ddPbsWX333XfWttTUVH322Wfq2LFjluskJyerX79+CgwMlKurqypWrKhJkybJMAybfikpKXrjjTfk6+urwoUL6+mnn9bvv/+e5Tb/+OMPvfTSS/Lz85Orq6sefvhhLViw4O4daBYqV66sxx9/PFN7enq6SpYsaRPYJ02apLp16+qBBx6Qu7u7wsLC9Nlnn912HyNGjJDFYsnUnnF6f2JiorXtyy+/VMuWLRUQECBXV1eVK1dOo0ePVlpamrVPo0aNtGrVKh05csR6um3GNcvZXdO9bt06NWjQQJ6enipSpIhat26t+Pj4LOs8dOiQunbtqiJFisjb21vdunXTlStXbnucGTp27KhvvvlG58+ft7Zt27ZNBw8ezPbzdP78efXt29f6eSpfvrzGjx9vHaFOTEyUr6+vJGnkyJGZTjPu2rWrChUqpISEBLVo0UKFCxdWp06drMv+eU13enq6pk+fripVqsjNzU2+vr5q1qyZtm/fbu3z3XffqX79+ipSpIgKFSqkihUr6q233srx67BkyRJVrFhRbm5uCgsL048//mhdtn79elksFn3++eeZ1ouOjpbFYtHmzZtvu4/WrVvL1dVVn376aaZttGvXTo6OjpnWuXHjhkaPHq1y5crJ1dVVQUFBeuutt5SSkmLTzzAMjRkzRg8++KA8PDz0+OOP63//+1+Wddzu/btbFi5cKIvFol27dmVa9u6778rR0VF//PGHpJu/J5UrV9aOHTtUt25dubu7q0yZMpo7d26mdVNSUjR8+HCVL19erq6uCgwM1IABAzK9JgBwvyF0A0A+CQoKUp06dbR06VJr2zfffKMLFy7ohRdeyNTfMAw9/fTTmjp1qpo1a6YpU6aoYsWKevPNNxUVFWXTt0ePHpo2bZqefPJJjRs3Ts7OzmrZsmWmbZ48eVKPPvqovv/+e0VGRmr69OkqX768unfvfkfXLl+6dElnzpzJ9Mj440D79u31448/6sSJEzbr/fTTTzp27JjN8U+fPl3Vq1fXqFGj9O6778rJyUnPP/+8Vq1alef6/mnRokUqVKiQoqKiNH36dIWFhWnYsGEaNGiQtc/bb7+t0NBQ+fj46KOPPtJHH310y9fo+++/V3h4uE6dOqURI0YoKipKmzZtUr169WwCf4Z27drp0qVLGjt2rNq1a6dFixZZT+vOiWeffVYWi0UrV660tkVHR+uhhx5SjRo1MvW/cuWKGjZsqI8//lgRERGaMWOG6tWrp8GDB1s/T76+vpozZ44k6ZlnnrEe97PPPmvdzo0bNxQeHq7ixYtr0qRJatu2bbY1du/e3RoSx48fr0GDBsnNzU1btmyRJP3vf//TU089pZSUFI0aNUqTJ0/W008/rY0bN+boNfjhhx/Ut29fvfjiixo1apTOnj2rZs2a6ZdffpF0MxAGBgZqyZIlmdZdsmSJypUrl6NLIDw8PNS6dWub393du3frf//7X7Z/4OjRo4eGDRumGjVqaOrUqWrYsKHGjh2b6Xd92LBhGjp0qKpVq6aJEyeqbNmyevLJJ5WcnGzTLyfv393y3HPPyd3dPdvXrVGjRipZsqS17c8//1SLFi0UFhamCRMm6MEHH9Srr75q88e89PR0Pf3005o0aZJatWqlmTNnqk2bNpo6darat29/V+sHgHuOAQAw1cKFCw1JxrZt24xZs2YZhQsXNq5cuWIYhmE8//zzxuOPP24YhmGULl3aaNmypXW9L774wpBkjBkzxmZ7zz33nGGxWIxDhw4ZhmEYcXFxhiTjtddes+nXsWNHQ5IxfPhwa1v37t2NEiVKGGfOnLHp+8ILLxje3t7Wug4fPmxIMhYuXHjLY1u/fr0hKdvH8ePHDcMwjP379xuSjJkzZ9qs/9prrxmFChWy7tcwDJufDcMwUlNTjcqVKxuNGze2aS9durTRpUsX6/Phw4cbWf1vLeP1P3z4cLb7MAzDeOWVVwwPDw/j2rVr1raWLVsapUuXztQ3q9cnNDTUKF68uHH27Flr2+7duw0HBwcjIiIiU50vvfSSzTafeeYZ44EHHsi0r3/q0qWL4enpaRjGzc9CkyZNDMMwjLS0NMPf398YOXKktb6JEyda1xs9erTh6elpHDhwwGZ7gwYNMhwdHY2kpCTDMAzj9OnTmT43f9+3JGPQoEFZLvv7a7Vu3TpDktGnT59MfdPT0w3DMIypU6cakozTp0/f9rj/KeMztn37dmvbkSNHDDc3N+OZZ56xtg0ePNhwdXU1zp8/b207deqU4eTklOUx/l3G5/vTTz81vv76a8NisVhfpzfffNMoW7asYRiG0bBhQ+Phhx+2rpfxO9mjRw+b7fXv39+QZKxbt85ah4uLi9GyZUvra2IYhvHWW28Zkmw+3zl9/zJem9sd2999+umnhiRj/fr11rYOHToYAQEBRlpamrVt586dmT73DRs2NCQZkydPtralpKRYfx9SU1MNwzCMjz76yHBwcDA2bNhgs++5c+cakoyNGzfmuF4AKGgY6QaAfNSuXTtdvXpVX3/9tS5duqSvv/4625Gy1atXy9HRUX369LFp79evnwzD0DfffGPtJylTv39O/mUYhlasWKFWrVrJMAybEenw8HBduHBBO3fuzNNxDRs2TN99912mR7FixSRJFSpUUGhoqJYvX25dJy0tTZ999platWold3d3a/vff/7zzz914cIFNWjQIM+1ZeXv+8gYpW/QoIGuXLmiX3/9NdfbO378uOLi4tS1a1frMUs3J6J64oknrO/R3/Xq1cvmeYMGDXT27FldvHgxx/vt2LGjYmNjdeLECa1bt04nTpzI9vP06aefqkGDBipatKjNe9+0aVOlpaXZnJZ9O6+++upt+6xYsUIWi0XDhw/PtCzjMoAiRYpIunm6f15Oka5Tp47CwsKsz0uVKqXWrVtr7dq11ksFIiIilJKSYnOJwvLly3Xjxo1ME6PdypNPPqlixYpp2bJlMgxDy5YtU4cOHbLsm/F+/3MEul+/fpJkPWvj+++/V2pqqv7v//7P5tKIf/7uSnf3/cuJiIgIHTt2TOvXr7e2LVmyRO7u7pnObnByctIrr7xife7i4qJXXnlFp06d0o4dO6z1h4SE6KGHHrKpv3HjxpJksx8AuN8wkRoA5CNfX181bdpU0dHRunLlitLS0rKdgOzIkSMKCAhQ4cKFbdozZhc+cuSI9b8ODg4qV66cTb+KFSvaPD99+rTOnz+vefPmad68eVnu89SpU3k6ripVqqhp06a37NO+fXu99dZb+uOPP1SyZEnFxsbq1KlTmU4t/frrrzVmzBjFxcXZXOuZ1fXaefW///1PQ4YM0bp16zKF3AsXLuR6exnvxT9fc+nm+7V27VrrhGMZSpUqZdOvaNGikm7+ocHLyytH+824rnr58uWKi4vTI488ovLly2d5OvvBgwe1Z88e6zXb/5TT997JyUkPPvjgbfslJCQoICDA5o8Q/9S+fXu9//776tGjhwYNGqQmTZro2Wef1XPPPScHh9uPCwQHB2dqq1Chgq5cuaLTp0/L399fDz30kB555BEtWbJE3bt3l3QzPD766KMqX778bfeRwdnZWc8//7yio6NVq1YtHT16NNs/cGT8Tv5z+/7+/ipSpIjN725Wx+Hr62v9PGS4W+9fTj3xxBMqUaKElixZoiZNmig9PV1Lly5V69atM30nBQQE2Hy2pZvvg3RznoBHH31UBw8eVHx8fL7VDwD3EkI3AOSzjh07qmfPnjpx4oSaN29uHe0zW8ZI4osvvqguXbpk2cfMWwS1b99egwcP1qeffqq+ffvqk08+kbe3t5o1a2bts2HDBj399NN67LHH9N5776lEiRJydnbWwoULFR0dfcvtZxfK/z45mnRzMqqGDRvKy8tLo0aNUrly5eTm5qadO3dq4MCB+Xbbq6wm35KUaZK8W3F1ddWzzz6rxYsX67fffrvlfZXT09P1xBNPaMCAAVkuzwhJOdlnTgJxTri7u+vHH3/U+vXrtWrVKq1Zs0bLly9X48aN9e2332b7GuVWRESEXn/9df3+++9KSUnRli1bNGvWrFxvp2PHjpo7d65GjBihatWqqVKlSrfsfzf/UHS33r+ccnR0VMeOHTV//ny999572rhxo44dO5arswP+Lj09XVWqVNGUKVOyXB4YGHgn5QLAPY3QDQD57JlnntErr7yiLVu22Jxu/U+lS5fW999/r0uXLtmMLGWc/ly6dGnrf9PT05WQkGAz0rp//36b7WXMbJ6WlnbbUWkzlClTRrVq1dLy5csVGRmplStXqk2bNjb3J1+xYoXc3Ny0du1am/aFCxfedvsZI4Pnz5+3+UNGxmhihtjYWJ09e1YrV67UY489Zm0/fPhwpm3mNDRlvBf/fM2lm++Xj49PppHAu6Vjx45asGCBHBwcspyQL0O5cuV0+fLl2773dysolitXTmvXrtW5c+duOdrt4OCgJk2aqEmTJpoyZYreffddvf3221q/fv1taz148GCmtgMHDsjDw8NmRPWFF15QVFSUli5dar2/el4m76pfv75KlSql2NhYjR8/Ptt+Gb+TBw8etLnv9cmTJ3X+/Hmb392M4yhbtqy13+nTpzPdgiyn79/dFBERocmTJ+u///2vvvnmG/n6+io8PDxTv2PHjmU6k+PAgQOSZJ3Rvly5ctq9e7eaNGlyV/8YAQAFAdd0A0A+K1SokObMmaMRI0aoVatW2fZr0aKF0tLSMo3ITZ06VRaLRc2bN5ck639nzJhh0++fM207Ojqqbdu2WrFihXV25787ffp0Xg4nV9q3b68tW7ZowYIFOnPmTKbg4+joKIvFYjM6nZiYqC+++OK22844vf7v17YmJydr8eLFmfYh2Y4op6am6r333su0TU9Pzxydbl6iRAmFhoZq8eLFNrfw+uWXX/Ttt9+qRYsWt91GXj3++OMaPXq0Zs2aJX9//2z7tWvXTps3b9batWszLTt//rxu3Lgh6eZM3Rltd6Jt27YyDCPLGdkzXvtz585lWhYaGipJObqN1ObNm22u9T969Ki+/PJLPfnkkzaj5D4+PmrevLk+/vhjLVmyRM2aNbO533ZOWSwWzZgxQ8OHD1fnzp2z7Zfxfv/zdzBjlDfjzgJNmzaVs7OzZs6cafN5zGqW/Jy+f3dT1apVVbVqVb3//vtasWKFXnjhBTk5ZR6vuXHjhv7zn/9Yn6empuo///mPfH19rdfct2vXTn/88Yfmz5+faf2rV69mmq0dAO4njHQDgB1kd3r337Vq1UqPP/643n77bSUmJqpatWr69ttv9eWXX6pv377WkBkaGqoOHTrovffe04ULF1S3bl3FxMTo0KFDmbY5btw4rV+/XrVr11bPnj1VqVIlnTt3Tjt37tT333+fZQjKiQ0bNujatWuZ2jP+0Z6hXbt26t+/v/r3769ixYplGrVr2bKlpkyZombNmqljx446deqUZs+erfLly2vPnj23rOHJJ59UqVKl1L17d7355ptydHTUggUL5Ovrq6SkJGu/unXrqmjRourSpYv69Okji8Wijz76KMvTusPCwrR8+XJFRUXpkUceUaFChbL9Q8nEiRPVvHlz1alTR927d9fVq1c1c+ZMeXt73/K07zvl4OCgIUOG3Lbfm2++qa+++kpPPfWUunbtqrCwMCUnJ2vv3r367LPPlJiYKB8fH7m7u6tSpUpavny5KlSooGLFiqly5cqqXLlyrup6/PHH1blzZ82YMUMHDx5Us2bNlJ6erg0bNujxxx9XZGSkRo0apR9//FEtW7ZU6dKlderUKb333nt68MEHVb9+/dvuo3LlygoPD1efPn3k6upq/cNJVkE/IiLCOn/C6NGjc3Usf9e6dWu1bt36ln2qVaumLl26aN68edbLGbZu3arFixerTZs21nvW+/r6qn///ho7dqyeeuoptWjRQrt27dI333yT6Y8COX3/7raIiAj1799fkrI9tTwgIEDjx49XYmKiKlSoYJ1jYN68eXJ2dpYkde7cWZ988ol69eql9evXq169ekpLS9Ovv/6qTz75RGvXrlXNmjXvev0AcE+w17TpAPBv8fdbht3KP28ZZhiGcenSJeONN94wAgICDGdnZyM4ONiYOHGize2FDMMwrl69avTp08d44IEHDE9PT6NVq1bG0aNHs7x10MmTJ43evXsbgYGBhrOzs+Hv7280adLEmDdvnrXP3bplWFa3LapXr16Wt1PK8MEHHxjBwcGGq6ur8dBDDxkLFy7M8nZg/7xlmGEYxo4dO4zatWsbLi4uRqlSpYwpU6ZkecuwjRs3Go8++qjh7u5uBAQEGAMGDDDWrl2b6bZJly9fNjp27GgUKVLEkGS9JVZ2r8/3339v1KtXz3B3dze8vLyMVq1aGfv27bPpk3Es/7xNVlZ1ZuXvtwzLTla3DDOMm5+nwYMHG+XLlzdcXFwMHx8fo27dusakSZOst3YyDMPYtGmTERYWZri4uNi8j7fa9z9vGWYYhnHjxg1j4sSJxkMPPWS4uLgYvr6+RvPmzY0dO3YYhmEYMTExRuvWrY2AgADDxcXFCAgIMDp06JDptlhZkWT07t3b+Pjjj62fl+rVq9u8f3+XkpJiFC1a1PD29jauXr162+0bhu0tw27ln7cMMwzDuH79ujFy5EijTJkyhrOzsxEYGGgMHjzY5pZ0hnHzVm8jR440SpQoYbi7uxuNGjUyfvnllyw/3zl9/7L73ctOVrcMy3D8+HHD0dHRqFChwi2Pffv27UadOnUMNzc3o3Tp0sasWbMy9U1NTTXGjx9vPPzww4arq6tRtGhRIywszBg5cqRx4cKFHNcLAAWNxTByMWMLAABAAXTjxg0FBASoVatW+uCDD+xdToFx5swZlShRQsOGDdPQoUMzLW/UqJHOnDmT5SUrAICbuKYbAADc97744gudPn1aERER9i6lQFm0aJHS0tJueQ07AODWuKYbAADct37++Wft2bNHo0ePVvXq1dWwYUN7l1QgrFu3Tvv27dM777yjNm3aWGchBwDkHqEbAADct+bMmaOPP/5YoaGhWrRokb3LKTBGjRqlTZs2qV69epo5c6a9ywGAAo1rugEAAAAAMAnXdAMAAAAAYBJCNwAAAAAAJikQ13Snp6fr2LFjKly4sCwWi73LAQAAAAD8yxmGoUuXLikgIEAODtmPZxeI0H3s2DEFBgbauwwAAAAAAGwcPXpUDz74YLbLC0ToLly4sKSbB+Pl5WXnagAAAAAA/3YXL15UYGCgNa9mp0CE7oxTyr28vAjdAAAAAIB7xu0ugWYiNQAAAAAATELoBgAAAADAJIRuAAAAAABMUiCu6QYAAACAgiwtLU3Xr1+3dxnIBWdnZzk6Ot7xdgjdAAAAAGASwzB04sQJnT9/3t6lIA+KFCkif3//206WdiuEbgAAAAAwSUbgLl68uDw8PO4ovCH/GIahK1eu6NSpU5KkEiVK5HlbhG4AAAAAMEFaWpo1cD/wwAP2Lge55O7uLkk6deqUihcvnudTzZlIDQAAAABMkHENt4eHh50rQV5lvHd3cj0+oRsAAAAATMQp5QXX3XjvCN0AAAAAAJiE0A0AAAAAgEmYSA0AAAAA8lnQoFX5ur/EcS3zdX/4CyPdAAAAAIB73p1MZmZPhG4AAAAAQCZr1qxR/fr1VaRIET3wwAN66qmnlJCQYF3++++/q0OHDipWrJg8PT1Vs2ZN/fzzz9bl//3vf/XII4/Izc1NPj4+euaZZ6zLLBaLvvjiC5v9FSlSRIsWLZIkJSYmymKxaPny5WrYsKHc3Ny0ZMkSnT17Vh06dFDJkiXl4eGhKlWqaOnSpTbbSU9P14QJE1S+fHm5urqqVKlSeueddyRJjRs3VmRkpE3/06dPy8XFRTExMXfjZcuE0A0AAAAAyCQ5OVlRUVHavn27YmJi5ODgoGeeeUbp6em6fPmyGjZsqD/++ENfffWVdu/erQEDBig9PV2StGrVKj3zzDNq0aKFdu3apZiYGNWqVSvXNQwaNEivv/664uPjFR4ermvXriksLEyrVq3SL7/8opdfflmdO3fW1q1bresMHjxY48aN09ChQ7Vv3z5FR0fLz89PktSjRw9FR0crJSXF2v/jjz9WyZIl1bhx4zt8xbLGNd0AAAAAgEzatm1r83zBggXy9fXVvn37tGnTJp0+fVrbtm1TsWLFJEnly5e39n3nnXf0wgsvaOTIkda2atWq5bqGvn376tlnn7Vp69+/v/Xn//u//9PatWv1ySefqFatWrp06ZKmT5+uWbNmqUuXLpKkcuXKqX79+pKkZ599VpGRkfryyy/Vrl07SdKiRYvUtWtX027txkg3AAAAACCTgwcPqkOHDipbtqy8vLwUFBQkSUpKSlJcXJyqV69uDdz/FBcXpyZNmtxxDTVr1rR5npaWptGjR6tKlSoqVqyYChUqpLVr1yopKUmSFB8fr5SUlGz37ebmps6dO2vBggWSpJ07d+qXX35R165d77jW7DDSDQAAAADIpFWrVipdurTmz5+vgIAApaenq3LlykpNTZW7u/st173dcovFIsMwbNqymijN09PT5vnEiRM1ffp0TZs2TVWqVJGnp6f69u2r1NTUHO1XunmKeWhoqH7//XctXLhQjRs3VunSpW+7Xl4x0g0AAAAAsHH27Fnt379fQ4YMUZMmTRQSEqI///zTurxq1aqKi4vTuXPnsly/atWqt5yYzNfXV8ePH7c+P3jwoK5cuXLbujZu3KjWrVvrxRdfVLVq1VS2bFkdOHDAujw4OFju7u633HeVKlVUs2ZNzZ8/X9HR0XrppZduu987QegGAAAAANgoWrSoHnjgAc2bN0+HDh3SunXrFBUVZV3eoUMH+fv7q02bNtq4caN+++03rVixQps3b5YkDR8+XEuXLtXw4cMVHx+vvXv3avz48db1GzdurFmzZmnXrl3avn27evXqJWdn59vWFRwcrO+++06bNm1SfHy8XnnlFZ08edK63M3NTQMHDtSAAQP04YcfKiEhQVu2bNEHH3xgs50ePXpo3LhxMgzDZlZ1MxC6AQAAAAA2HBwctGzZMu3YsUOVK1fWG2+8oYkTJ1qXu7i46Ntvv1Xx4sXVokULValSRePGjZOjo6MkqVGjRvr000/11VdfKTQ0VI0bN7aZYXzy5MkKDAxUgwYN1LFjR/Xv318eHh63rWvIkCGqUaOGwsPD1ahRI2vw/7uhQ4eqX79+GjZsmEJCQtS+fXudOnXKpk+HDh3k5OSkDh06yM3N7Q5eqduzGP88kf4edPHiRXl7e+vChQvy8vKydzkAAOA+NLvXOnuXkCe955pzixsAd+7atWs6fPiwypQpY3qwQ+4kJiaqXLly2rZtm2rUqJFtv1u9hznNqUykBgAAAAD4V7h+/brOnj2rIUOG6NFHH71l4L5bOL0cAAAAAPCvsHHjRpUoUULbtm3T3Llz82WfjHQDAAAAAP4VGjVqlOlWZWZjpBsAAAAAAJMQugEAAAAAMAmhGwAAAAAAkxC6AQAAAAAwCaEbAAAAAACTELoBAAAAADAJoRsAAAAAYHexsbGyWCw6f/78Xe1rb9ynGwAAAADy2wjvfN7fhfzdXx7UrVtXx48fl7f37V+b3PS1N0a6AQAAAAB3JDU19Y634eLiIn9/f1kslrva194I3QAAAAAAG40aNVJkZKQiIyPl7e0tHx8fDR06VIZhSJKCgoI0evRoRUREyMvLSy+//LIk6aefflKDBg3k7u6uwMBA9enTR8nJydbtpqSkaODAgQoMDJSrq6vKly+vDz74QFLmU8aPHDmiVq1aqWjRovL09NTDDz+s1atXZ9lXklasWKGHH35Yrq6uCgoK0uTJk22OKSgoSO+++65eeuklFS5cWKVKldK8efPMegmtCN0AAAAAgEwWL14sJycnbd26VdOnT9eUKVP0/vvvW5dPmjRJ1apV065duzR06FAlJCSoWbNmatu2rfbs2aPly5frp59+UmRkpHWdiIgILV26VDNmzFB8fLz+85//qFChQlnuv3fv3kpJSdGPP/6ovXv3avz48dn23bFjh9q1a6cXXnhBe/fu1YgRIzR06FAtWrTIpt/kyZNVs2ZN7dq1S6+99ppeffVV7d+//85frFvgmm4AAAAAQCaBgYGaOnWqLBaLKlasqL1792rq1Knq2bOnJKlx48bq16+ftX+PHj3UqVMn9e3bV5IUHBysGTNmqGHDhpozZ46SkpL0ySef6LvvvlPTpk0lSWXLls12/0lJSWrbtq2qVKly275TpkxRkyZNNHToUElShQoVtG/fPk2cOFFdu3a19mvRooVee+01SdLAgQM1depUrV+/XhUrVsz9C5RDjHQDAAAAADJ59NFHba6ZrlOnjg4ePKi0tDRJUs2aNW367969W4sWLVKhQoWsj/DwcKWnp+vw4cOKi4uTo6OjGjZsmKP99+nTR2PGjFG9evU0fPhw7dmzJ9u+8fHxqlevnk1bvXr1bOqVpKpVq1p/tlgs8vf316lTp3JUT14RugEAAAAAuebp6Wnz/PLly3rllVcUFxdnfezevVsHDx5UuXLl5O7unqvt9+jRQ7/99ps6d+6svXv3qmbNmpo5c+Yd1ezs7Gzz3GKxKD09/Y62eTuEbgAAAABAJj///LPN8y1btig4OFiOjo5Z9q9Ro4b27dun8uXLZ3q4uLioSpUqSk9P1w8//JDjGgIDA9WrVy+tXLlS/fr10/z587PsFxISoo0bN9q0bdy4URUqVMi23vxC6AYAAAAAZJKUlKSoqCjt379fS5cu1cyZM/X6669n23/gwIHatGmTIiMjFRcXp4MHD+rLL7+0TqQWFBSkLl266KWXXtIXX3yhw4cPKzY2Vp988kmW2+vbt6/Wrl2rw4cPa+fOnVq/fr1CQkKy7NuvXz/FxMRo9OjROnDggBYvXqxZs2apf//+d/5C3CEmUgMAAAAAZBIREaGrV6+qVq1acnR01Ouvv269NVhWqlatqh9++EFvv/22GjRoIMMwVK5cObVv397aZ86cOXrrrbf02muv6ezZsypVqpTeeuutLLeXlpam3r176/fff5eXl5eaNWumqVOnZtm3Ro0a+uSTTzRs2DCNHj1aJUqU0KhRo2wmUbMXi5Fxo7V72MWLF+Xt7a0LFy7Iy8vL3uUAAID70Oxe6+xdQp70ntvY3iUAyMa1a9d0+PBhlSlTRm5ubvYuJ1caNWqk0NBQTZs2zd6l2NWt3sOc5lROLwcAAAAAwCSEbgAAAAAATMI13QAAAAAAG7GxsfYu4b7BSDcAAAAAACYhdAMAAAAAYBJCNwAAAAAAJiF0AwAAAABgEkI3AAAAAAAmIXQDAAAAAGASQjcAAAAAwO5GjBih0NBQ6/OuXbuqTZs2dqvnbuE+3QAAAACQz6osrpKv+9vbZW++7g9/YaQbAAAAAHBLqamp9i6hwCJ0AwAAAABsNGrUSJGRkerbt698fHwUHh6uX375Rc2bN1ehQoXk5+enzp0768yZM9Z10tPTNWHCBJUvX16urq4qVaqU3nnnHevygQMHqkKFCvLw8FDZsmU1dOhQXb9+3R6Hl68I3QAAAACATBYvXiwXFxdt3LhR48aNU+PGjVW9enVt375da9as0cmTJ9WuXTtr/8GDB2vcuHEaOnSo9u3bp+joaPn5+VmXFy5cWIsWLdK+ffs0ffp0zZ8/X1OnTrXHoeUrrukGAAAAAGQSHBysCRMmSJLGjBmj6tWr691337UuX7BggQIDA3XgwAGVKFFC06dP16xZs9SlSxdJUrly5VS/fn1r/yFDhlh/DgoKUv/+/bVs2TINGDAgn47IPgjdAAAAAIBMwsLCrD/v3r1b69evV6FChTL1S0hI0Pnz55WSkqImTZpku73ly5drxowZSkhI0OXLl3Xjxg15eXmZUvu9hNANAAAAAMjE09PT+vPly5fVqlUrjR8/PlO/EiVK6LfffrvltjZv3qxOnTpp5MiRCg8Pl7e3t5YtW6bJkyff9brvNYRuAAAAAMAt1ahRQytWrFBQUJCcnDLHyODgYLm7uysmJkY9evTItHzTpk0qXbq03n77bWvbkSNHTK35XsFEagAAAACAW+rdu7fOnTunDh06aNu2bUpISNDatWvVrVs3paWlyc3NTQMHDtSAAQP04YcfKiEhQVu2bNEHH3wg6WYoT0pK0rJly5SQkKAZM2bo888/t/NR5Q9CNwAAAADglgICArRx40alpaXpySefVJUqVdS3b18VKVJEDg43Y+XQoUPVr18/DRs2TCEhIWrfvr1OnTolSXr66af1xhtvKDIyUqGhodq0aZOGDh1qz0PKNxbDMIzcrjR79mxNnDhRJ06cULVq1TRz5kzVqlUr2/7Tpk3TnDlzlJSUJB8fHz333HMaO3as3NzccrS/ixcvytvbWxcuXPhXXGgPAADy3+xe6+xdQp70ntvY3iUAyMa1a9d0+PBhlSlTJsfZB/eWW72HOc2puR7pXr58uaKiojR8+HDt3LlT1apVU3h4uPUvGP8UHR2tQYMGafjw4YqPj9cHH3yg5cuX66233srtrgEAAAAAKFByHbqnTJminj17qlu3bqpUqZLmzp0rDw8PLViwIMv+mzZtUr169dSxY0cFBQXpySefVIcOHbR169Y7Lh4AAAAAgHtZrkJ3amqqduzYoaZNm/61AQcHNW3aVJs3b85ynbp162rHjh3WkP3bb79p9erVatGiRbb7SUlJ0cWLF20eAAAAAAAUNLm6ZdiZM2eUlpYmPz8/m3Y/Pz/9+uuvWa7TsWNHnTlzRvXr15dhGLpx44Z69ep1y9PLx44dq5EjR+amNAAAAAAA7jmmz14eGxurd999V++995527typlStXatWqVRo9enS26wwePFgXLlywPo4ePWp2mQAAAAAA3HW5Gun28fGRo6OjTp48adN+8uRJ+fv7Z7nO0KFD1blzZ+sN0qtUqaLk5GS9/PLLevvtt63Ty/+dq6urXF1dc1MaAAAAAAD3nFyNdLu4uCgsLEwxMTHWtvT0dMXExKhOnTpZrnPlypVMwdrR0VGSlIe7lQEAAAAAUGDkaqRbkqKiotSlSxfVrFlTtWrV0rRp05ScnKxu3bpJkiIiIlSyZEmNHTtWktSqVStNmTJF1atXV+3atXXo0CENHTpUrVq1soZvAAAAAADuR7kO3e3bt9fp06c1bNgwnThxQqGhoVqzZo11crWkpCSbke0hQ4bIYrFoyJAh+uOPP+Tr66tWrVrpnXfeuXtHAQAAAADAPchiFIBzvC9evChvb29duHBBXl5e9i4HAADch2b3WmfvEvKk99zG9i4BQDauXbumw4cPq0yZMnJzc7N3ObliGIZeeeUVffbZZ/rzzz+1a9cuhYaG2rusfHer9zCnOTXXI90AAAAAgDsT/1BIvu4v5Nf4XPVfs2aNFi1apNjYWJUtW1YHDhxQq1attGPHDh0/flyff/652rRpY06x9xnTbxkGAAAAAChYEhISVKJECdWtW1f+/v5KTk5WtWrVNHv2bHuXdlupqan2LsEGoRsAAAAAYNW1a1f93//9n5KSkmSxWBQUFKTmzZtrzJgxeuaZZ/K0zffee0/BwcFyc3OTn5+fnnvuOeuy9PR0TZgwQeXLl5erq6tKlSplMwfY3r171bhxY7m7u+uBBx7Qyy+/rMuXL9vU26ZNG73zzjsKCAhQxYoVJUlHjx5Vu3btVKRIERUrVkytW7dWYmJi3l6UO8Dp5QAAAAAAq+nTp6tcuXKaN2+etm3bdsd3ndq+fbv69Omjjz76SHXr1tW5c+e0YcMG6/LBgwdr/vz5mjp1qurXr6/jx4/r119/lSQlJycrPDxcderU0bZt23Tq1Cn16NFDkZGRWrRokXUbMTEx8vLy0nfffSdJun79unW9DRs2yMnJSWPGjFGzZs20Z88eubi43NEx5QahGwAAAABg5e3trcKFC8vR0VH+/v53vL2kpCR5enrqqaeeUuHChVW6dGlVr15dknTp0iVNnz5ds2bNUpcuXSRJ5cqVU/369SVJ0dHRunbtmj788EN5enpKkmbNmqVWrVpp/Pjx1rtoeXp66v3337eG6Y8//ljp6el6//33ZbFYJEkLFy5UkSJFFBsbqyeffPKOjyunOL0cAAAAAGCaJ554QqVLl1bZsmXVuXNnLVmyRFeuXJEkxcfHKyUlRU2aNMly3fj4eFWrVs0auCWpXr16Sk9P1/79+61tVapUsRm93r17tw4dOqTChQurUKFCKlSokIoVK6Zr164pISHBpCPNGiPdAAAAAADTFC5cWDt37lRsbKy+/fZbDRs2TCNGjNC2bdvk7u5+V/bx91AuSZcvX1ZYWJiWLFmSqa+vr+9d2WdOMdINAAAAADCVk5OTmjZtqgkTJmjPnj1KTEzUunXrFBwcLHd3d8XExGS5XkhIiHbv3q3k5GRr28aNG+Xg4GCdMC0rNWrU0MGDB1W8eHGVL1/e5uHt7X3Xj+9WCN0AAAAAgFu6fPmy4uLiFBcXJ0k6fPiw4uLilJSUdNt1v/76a82YMUNxcXE6cuSIPvzwQ6Wnp6tixYpyc3PTwIEDNWDAAH344YdKSEjQli1b9MEHH0iSOnXqJDc3N3Xp0kW//PKL1q9fr//7v/9T586drddzZ6VTp07y8fFR69attWHDBh0+fFixsbHq06ePfv/997vymuQUp5cDAAAAAG5p+/btevzxx63Po6KiJEldunSxmUU8K0WKFNHKlSs1YsQIXbt2TcHBwVq6dKkefvhhSdLQoUPl5OSkYcOG6dixYypRooR69eolSfLw8NDatWv1+uuv65FHHpGHh4fatm2rKVOm3HKfHh4e+vHHHzVw4EA9++yzunTpkkqWLKkmTZrIy8vrDl6J3LMYhmHk6x7z4OLFi/L29taFCxfy/QUCAAD/DrN7rbN3CXnSe25je5cAIBvXrl3T4cOHVaZMGbm5udm7HOTBrd7DnOZUTi8HAAAAAMAkhG4AAAAAQJ5t2LDBeluurB7/dlzTDQAAAADIs5o1a1onWENmhG4AAAAAQJ65u7urfPny9i7jnsXp5QAAAAAAmITQDQAAAACASQjdAAAAAACYhNANAAAAAIBJCN0AAAAAAJiE0A0AAAAAsNGoUSP17dvX3mXcF7hlGAAAAADks9m91uXr/nrPbZyv+8NfGOkGAAAAAMAkhG4AAAAAQLb+/PNPRUREqGjRovLw8FDz5s118OBB6/IjR46oVatWKlq0qDw9PfXwww9r9erV1nU7deokX19fubu7Kzg4WAsXLrTXodgFp5cDAAAAALLVtWtXHTx4UF999ZW8vLw0cOBAtWjRQvv27ZOzs7N69+6t1NRU/fjjj/L09NS+fftUqFAhSdLQoUO1b98+ffPNN/Lx8dGhQ4d09epVOx9R/iJ0AwAAAACylBG2N27cqLp160qSlixZosDAQH3xxRd6/vnnlZSUpLZt26pKlSqSpLJly1rXT0pKUvXq1VWzZk1JUlBQUL4fg71xejkAAAAAIEvx8fFycnJS7dq1rW0PPPCAKlasqPj4eElSnz59NGbMGNWrV0/Dhw/Xnj17rH1fffVVLVu2TKGhoRowYIA2bdqU78dgb4RuAAAAAECe9ejRQ7/99ps6d+6svXv3qmbNmpo5c6YkqXnz5jpy5IjeeOMNHTt2TE2aNFH//v3tXHH+InQDAAAAALIUEhKiGzdu6Oeff7a2nT17Vvv371elSpWsbYGBgerVq5dWrlypfv36af78+dZlvr6+6tKliz7++GNNmzZN8+bNy9djsDeu6QYAAAAAZCk4OFitW7dWz5499Z///EeFCxfWoEGDVLJkSbVu3VqS1LdvXzVv3lwVKlTQn3/+qfXr1yskJESSNGzYMIWFhenhhx9WSkqKvv76a+uyfwtCN4Bcm91rnb1LyLPecxvbuwQAAIACZeHChXr99df11FNPKTU1VY899phWr14tZ2dnSVJaWpp69+6t33//XV5eXmrWrJmmTp0qSXJxcdHgwYOVmJgod3d3NWjQQMuWLbPn4eQ7QjcAAAAA5LN7fSAgNjbW+nPRokX14YcfZts34/rtrAwZMkRDhgy5m6UVOFzTDQAAAACASQjdAAAAAACYhNANAAAAAIBJCN0AAAAAAJiEidQAAAAAmK7K4ir2LiFP9nbZa+8SUMAx0g0AAAAAJkpPT7d3Cciju/HeMdINAAAAACZwcXGRg4ODjh07Jl9fX7m4uMhisdi7LOSAYRhKTU3V6dOn5eDgIBcXlzxvi9ANAAAAACZwcHBQmTJldPz4cR07dsze5SAPPDw8VKpUKTk45P0kcUI3AAAAAJjExcVFpUqV0o0bN5SWlmbvcpALjo6OcnJyuuOzEwjdAAAAAGAii8UiZ2dnOTs727sU2AETqQEAAAAAYBJCNwAAAAAAJiF0AwAAAABgEkI3AAAAAAAmYSI1AAAAALgPze61zt4l5EnvuY3tXcJdxUg3AAAAAAAmIXQDAAAAAGASTi8HAAAAgGzEPxRi7xLyrtFse1cAMdINAAAAAIBpCN0AAAAAAJiE0A0AAAAAgEkI3QAAAAAAmITQDQAAAACASQjdAAAAAACYhFuGAQAA4F8naNAqe5eQJ4njWtq7BAC5xEg3AAAAAAAmIXQDAAAAAGASQjcAAAAAACbhmm4AAACgoBjhbe8K8q5MKXtXANgFI90AAAAAAJiE0A0AAAAAgEkI3QAAAAAAmITQDQAAAACASQjdAAAAAACYhNANAAAAAIBJCN0AAAAAAJiE0A0AAAAAgEkI3QAAAAAAmITQDQAAAACASQjdAAAAAACYhNANAAAAAIBJCN0AAAAAAJiE0A0AAAAAgEkI3QAAAAAAmITQDQAAAACASQjdAAAAAACYhNANAAAAAIBJCN0AAAAAAJiE0A0AAAAAgEkI3QAAAAAAmITQDQAAAACASQjdAAAAAACYhNANAAAAAIBJCN0AAAAAAJiE0A0AAAAAgEkI3QAAAAAAmITQDQAAAACASQjdAAAAAACYhNANAAAAAIBJCN0AAAAAAJiE0A0AAAAAgEkI3QAAAAAAmITQDQAAAACASQjdAAAAAACYxMneBQAAYJoR3vauIO9GXLB3BQAA4C5gpBsAAAAAAJPkKXTPnj1bQUFBcnNzU+3atbV169Zb9j9//rx69+6tEiVKyNXVVRUqVNDq1avzVDAAAAAAAAVFrk8vX758uaKiojR37lzVrl1b06ZNU3h4uPbv36/ixYtn6p+amqonnnhCxYsX12effaaSJUvqyJEjKlKkyN2oHwAAAACAe1auQ/eUKVPUs2dPdevWTZI0d+5crVq1SgsWLNCgQYMy9V+wYIHOnTunTZs2ydnZWZIUFBR0Z1UDAAAAAFAA5Or08tTUVO3YsUNNmzb9awMODmratKk2b96c5TpfffWV6tSpo969e8vPz0+VK1fWu+++q7S0tGz3k5KSoosXL9o8AAAAAAAoaHIVus+cOaO0tDT5+fnZtPv5+enEiRNZrvPbb7/ps88+U1pamlavXq2hQ4dq8uTJGjNmTLb7GTt2rLy9va2PwMDA3JQJAAAAAMA9wfTZy9PT01W8eHHNmzdPYWFhat++vd5++23NnTs323UGDx6sCxcuWB9Hjx41u0wAAAAAAO66XF3T7ePjI0dHR508edKm/eTJk/L3989ynRIlSsjZ2VmOjo7WtpCQEJ04cUKpqalycXHJtI6rq6tcXV1zUxoAAAAAAPecXI10u7i4KCwsTDExMda29PR0xcTEqE6dOlmuU69ePR06dEjp6enWtgMHDqhEiRJZBm4AAAAAAO4XuT69PCoqSvPnz9fixYsVHx+vV199VcnJydbZzCMiIjR48GBr/1dffVXnzp3T66+/rgMHDmjVqlV699131bt377t3FAAAAAAA3INyfcuw9u3b6/Tp0xo2bJhOnDih0NBQrVmzxjq5WlJSkhwc/srygYGBWrt2rd544w1VrVpVJUuW1Ouvv66BAwfevaMAAAAAAOAelOvQLUmRkZGKjIzMcllsbGymtjp16mjLli152RUAAAAAAAWW6bOXAwAAAADwb0XoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATOJk7wIAAPe+oEGr7F1CniS62bsCAADwb8dINwAAAAAAJiF0AwAAAABgEkI3AAAAAAAmIXQDAAAAAGASQjcAAAAAACYhdAMAAAAAYBJCNwAAAAAAJiF0AwAAAABgEkI3AAAAAAAmIXQDAAAAAGASQjcAAAAAACZxsncBuIeM8LZ3BXkz4oK9KwAAAACALDHSDQAAAACASQjdAAAAAACYhNANAAAAAIBJCN0AAAAAAJiE0A0AAAAAgEkI3QAAAAAAmITQDQAAAACASQjdAAAAAACYxMneBdxvggatsncJeZboZu8KAAAAAOD+wkg3AAAAAAAmIXQDAAAAAGASQjcAAAAAACYhdAMAAAAAYBJCNwAAAAAAJiF0AwAAAABgEkI3AAAAAAAmIXQDAAAAAGASQjcAAAAAACYhdAMAAAAAYBJCNwAAAAAAJiF0AwAAAABgEkI3AAAAAAAmIXQDAAAAAGASQjcAAAAAACYhdAMAAAAAYBJCNwAAAAAAJiF0AwAAAABgEkI3AAAAAAAmIXQDAAAAAGASQjcAAAAAACYhdAMAAAAAYBJCNwAAAAAAJiF0AwAAAABgEkI3AAAAAAAmIXQDAAAAAGASQjcAAAAAACYhdAMAAAAAYBJCNwAAAAAAJiF0AwAAAABgEkI3AAAAAAAmIXQDAAAAAGASQjcAAAAAACYhdAMAAAAAYBJCNwAAAAAAJiF0AwAAAABgEkI3AAAAAAAmIXQDAAAAAGASQjcAAAAAACYhdAMAAAAAYBJCNwAAAAAAJiF0AwAAAABgEkI3AAAAAAAmIXQDAAAAAGASQjcAAAAAACYhdAMAAAAAYBJCNwAAAAAAJiF0AwAAAABgEkI3AAAAAAAmIXQDAAAAAGCSPIXu2bNnKygoSG5ubqpdu7a2bt2ao/WWLVsmi8WiNm3a5GW3AAAAAAAUKLkO3cuXL1dUVJSGDx+unTt3qlq1agoPD9epU6duuV5iYqL69++vBg0a5LlYAAAAAAAKklyH7ilTpqhnz57q1q2bKlWqpLlz58rDw0MLFizIdp20tDR16tRJI0eOVNmyZe+oYAAAAAAACopche7U1FTt2LFDTZs2/WsDDg5q2rSpNm/enO16o0aNUvHixdW9e/e8VwoAAAAAQAHjlJvOZ86cUVpamvz8/Gza/fz89Ouvv2a5zk8//aQPPvhAcXFxOd5PSkqKUlJSrM8vXryYmzIBAAAAALgnmDp7+aVLl9S5c2fNnz9fPj4+OV5v7Nix8vb2tj4CAwNNrBIAAAAAAHPkaqTbx8dHjo6OOnnypE37yZMn5e/vn6l/QkKCEhMT1apVK2tbenr6zR07OWn//v0qV65cpvUGDx6sqKgo6/OLFy8SvAEAAAAABU6uQreLi4vCwsIUExNjve1Xenq6YmJiFBkZman/Qw89pL1799q0DRkyRJcuXdL06dOzDdKurq5ydXXNTWkAAAAAANxzchW6JSkqKkpdunRRzZo1VatWLU2bNk3Jycnq1q2bJCkiIkIlS5bU2LFj5ebmpsqVK9usX6RIEUnK1A4AAAAAwP0m16G7ffv2On36tIYNG6YTJ04oNDRUa9assU6ulpSUJAcHUy8VBwAAAACgQMh16JakyMjILE8nl6TY2Nhbrrto0aK87BIAAAAAgAKHIWkAAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkTvYuAAAA3D/iHwqxdwl512i2vSsAANyHGOkGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABM4mTvAoB/s/iHQuxdQt40mm3vCgAAAIACgZFuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABM4mTvAgAAQGZVFlexdwl58om9CwAA4B7DSDcAAAAAACYhdAMAAAAAYBJOL0eBV1BPwZQ4DRMAAAC43zHSDQAAAACASQjdAAAAAACYhNANAAAAAIBJCN0AAAAAAJiE0A0AAAAAgEkI3QAAAAAAmITQDQAAAACASQjdAAAAAACYhNANAAAAAIBJCN0AAAAAAJiE0A0AAAAAgEkI3QAAAAAAmITQDQAAAACASQjdAAAAAACYhNANAAAAAIBJCN0AAAAAAJiE0A0AAAAAgEkI3QAAAAAAmITQDQAAAACASQjdAAAAAACYhNANAAAAAIBJCN0AAAAAAJiE0A0AAAAAgEkI3QAAAAAAmITQDQAAAACASfIUumfPnq2goCC5ubmpdu3a2rp1a7Z958+frwYNGqho0aIqWrSomjZtesv+AAAAAADcL3IdupcvX66oqCgNHz5cO3fuVLVq1RQeHq5Tp05l2T82NlYdOnTQ+vXrtXnzZgUGBurJJ5/UH3/8ccfFAwAAAABwL8t16J4yZYp69uypbt26qVKlSpo7d648PDy0YMGCLPsvWbJEr732mkJDQ/XQQw/p/fffV3p6umJiYu64eAAAAAAA7mW5Ct2pqanasWOHmjZt+tcGHBzUtGlTbd68OUfbuHLliq5fv65ixYpl2yclJUUXL160eQAAAAAAUNDkKnSfOXNGaWlp8vPzs2n38/PTiRMncrSNgQMHKiAgwCa4/9PYsWPl7e1tfQQGBuamTAAAAAAA7gn5Onv5uHHjtGzZMn3++edyc3PLtt/gwYN14cIF6+Po0aP5WCUAAAAAAHeHU246+/j4yNHRUSdPnrRpP3nypPz9/W+57qRJkzRu3Dh9//33qlq16i37urq6ytXVNTelAQAAAABwz8nVSLeLi4vCwsJsJkHLmBStTp062a43YcIEjR49WmvWrFHNmjXzXi0AAAAAAAVIrka6JSkqKkpdunRRzZo1VatWLU2bNk3Jycnq1q2bJCkiIkIlS5bU2LFjJUnjx4/XsGHDFB0draCgIOu134UKFVKhQoXu4qEAAAAAAHBvyXXobt++vU6fPq1hw4bpxIkTCg0N1Zo1a6yTqyUlJcnB4a8B9Dlz5ig1NVXPPfeczXaGDx+uESNG3Fn1AAAAAADcw3IduiUpMjJSkZGRWS6LjY21eZ6YmJiXXQAAAAAAUODl6+zlAAAAAAD8mxC6AQAAAAAwCaEbAAAAAACTELoBAAAAADAJoRsAAAAAAJMQugEAAAAAMAmhGwAAAAAAkxC6AQAAAAAwCaEbAAAAAACTELoBAAAAADAJoRsAAAAAAJMQugEAAAAAMAmhGwAAAAAAkxC6AQAAAAAwCaEbAAAAAACTELoBAAAAADAJoRsAAAAAAJMQugEAAAAAMAmhGwAAAAAAkxC6AQAAAAAwCaEbAAAAAACTELoBAAAAADAJoRsAAAAAAJMQugEAAAAAMAmhGwAAAAAAkxC6AQAAAAAwCaEbAAAAAACTELoBAAAAADAJoRsAAAAAAJMQugEAAAAAMAmhGwAAAAAAkxC6AQAAAAAwCaEbAAAAAACTELoBAAAAADAJoRsAAAAAAJMQugEAAAAAMAmhGwAAAAAAkxC6AQAAAAAwCaEbAAAAAACTELoBAAAAADAJoRsAAAAAAJMQugEAAAAAMAmhGwAAAAAAkxC6AQAAAAAwCaEbAAAAAACTELoBAAAAADAJoRsAAAAAAJMQugEAAAAAMAmhGwAAAAAAkxC6AQAAAAAwCaEbAAAAAACTELoBAAAAADAJoRsAAAAAAJMQugEAAAAAMAmhGwAAAAAAkxC6AQAAAAAwCaEbAAAAAACTELoBAAAAADAJoRsAAAAAAJMQugEAAAAAMAmhGwAAAAAAkxC6AQAAAAAwCaEbAAAAAACTELoBAAAAADAJoRsAAAAAAJMQugEAAAAAMAmhGwAAAAAAkxC6AQAAAAAwCaEbAAAAAACTELoBAAAAADAJoRsAAAAAAJMQugEAAAAAMAmhGwAAAAAAkxC6AQAAAAAwCaEbAAAAAACTELoBAAAAADAJoRsAAAAAAJMQugEAAAAAMAmhGwAAAAAAkxC6AQAAAAAwCaEbAAAAAACTELoBAAAAADAJoRsAAAAAAJMQugEAAAAAMAmhGwAAAAAAkxC6AQAAAAAwCaEbAAAAAACTELoBAAAAADAJoRsAAAAAAJMQugEAAAAAMAmhGwAAAAAAkxC6AQAAAAAwCaEbAAAAAACTELoBAAAAADAJoRsAAAAAAJMQugEAAAAAMAmhGwAAAAAAk+QpdM+ePVtBQUFyc3NT7dq1tXXr1lv2//TTT/XQQw/Jzc1NVapU0erVq/NULAAAAAAABUmuQ/fy5csVFRWl4cOHa+fOnapWrZrCw8N16tSpLPtv2rRJHTp0UPfu3bVr1y61adNGbdq00S+//HLHxQMAAAAAcC/LdeieMmWKevbsqW7duqlSpUqaO3euPDw8tGDBgiz7T58+Xc2aNdObb76pkJAQjR49WjVq1NCsWbPuuHgAAAAAAO5luQrdqamp2rFjh5o2bfrXBhwc1LRpU23evDnLdTZv3mzTX5LCw8Oz7Q8AAAAAwP3CKTedz5w5o7S0NPn5+dm0+/n56ddff81ynRMnTmTZ/8SJE9nuJyUlRSkpKdbnFy5ckCRdvHgxN+XaRXrKFXuXkGcXLYa9S8iTtKtp9i4hzy6nFczar6Ym27uEPCsI3yP3ooL63VZQv9ekgvvdVlC/16SC+93G91re8L2W//hey398r5kro07DuPXvZa5Cd34ZO3asRo4cmak9MDDQDtX8e3jbu4A8i7d3AXlWy94F5NWhp+1dQZ69udDeFSA/FdzvNamgfrcV2O81qcB+t/G99u/C91r+43st/xW077VLly7J2zv7385chW4fHx85Ojrq5MmTNu0nT56Uv79/luv4+/vnqr8kDR48WFFRUdbn6enpOnfunB544AFZLJbclAzckYsXLyowMFBHjx6Vl5eXvcsBgDvG9xqA+w3fa7AXwzB06dIlBQQE3LJfrkK3i4uLwsLCFBMTozZt2ki6GYhjYmIUGRmZ5Tp16tRRTEyM+vbta2377rvvVKdOnWz34+rqKldXV5u2IkWK5KZU4K7y8vLiSxzAfYXvNQD3G77XYA+3GuHOkOvTy6OiotSlSxfVrFlTtWrV0rRp05ScnKxu3bpJkiIiIlSyZEmNHTtWkvT666+rYcOGmjx5slq2bKlly5Zp+/btmjdvXm53DQAAAABAgZLr0N2+fXudPn1aw4YN04kTJxQaGqo1a9ZYJ0tLSkqSg8Nfk6LXrVtX0dHRGjJkiN566y0FBwfriy++UOXKle/eUQAAAAAAcA+yGLebag34F0tJSdHYsWM1ePDgTJc8AEBBxPcagPsN32u41xG6AQAAAAAwicPtuwAAAAAAgLwgdAMAAAAAYBJCNwAABVSjRo1sbskJAADuPYRuAADuc4RzAADsh9ANSEpNTbV3CQAAALhD/JsO9yJCN/6VGjVqpMjISPXt21c+Pj5ydXWVxWJRTEyMatasKQ8PD9WtW1f79++3rjNixAiFhobqo48+UlBQkLy9vfXCCy/o0qVLdjwSALjpvffeU3BwsNzc3OTn56fnnntOktS1a1f98MMPmj59uiwWiywWixITExUbGyuLxaK1a9eqevXqcnd3V+PGjXXq1Cl98803CgkJkZeXlzp27KgrV67Y+egA/FtdunRJnTp1kqenp0qUKKGpU6fanL0TFBSk0aNHKyIiQl5eXnr55Zet32/nz5+3bicuLs76/QfkN0I3/rUWL14sFxcXbdy4UXPnzpUkvf3225o8ebK2b98uJycnvfTSSzbrJCQk6IsvvtDXX3+tr7/+Wj/88IPGjRtnj/IBwGr79u3q06ePRo0apf3792vNmjV67LHHJEnTp09XnTp11LNnTx0/flzHjx9XYGCgdd0RI0Zo1qxZ2rRpk44ePap27dpp2rRpio6O1qpVq/Ttt99q5syZ9jo0AP9yUVFR2rhxo7766it999132rBhg3bu3GnTZ9KkSapWrZp27dqloUOH2qlSIHtO9i4AsJfg4GBNmDBBknT8+HFJ0jvvvKOGDRtKkgYNGqSWLVvq2rVrcnNzkySlp6dr0aJFKly4sCSpc+fOiomJ0TvvvGOHIwCAm5KSkuTp6amnnnpKhQsXVunSpVW9enVJkre3t1xcXOTh4SF/f/9M644ZM0b16tWTJHXv3l2DBw9WQkKCypYtK0l67rnntH79eg0cODD/DggAdHOUe/HixYqOjlaTJk0kSQsXLlRAQIBNv8aNG6tfv37W50ePHs3XOoHbYaQb/1phYWGZ2qpWrWr9uUSJEpKkU6dOWduCgoKsgTujz9+XA4A9PPHEEypdurTKli2rzp07a8mSJTk+Jfzv33t+fn7y8PCwBu6MNr7nANjDb7/9puvXr6tWrVrWNm9vb1WsWNGmX82aNfO7NCBXCN341/L09MzU5uzsbP3ZYrFIujm6ndXyjD5/Xw4A9lC4cGHt3LlTS5cuVYkSJTRs2DBVq1bN5nrG7Pzze4/vOQAFzT//TefgcDPiGIZhbbt+/Xq+1gT8HaEbAID7gJOTk5o2baoJEyZoz549SkxM1Lp16yRJLi4uSktLs3OFAJA7ZcuWlbOzs7Zt22Ztu3Dhgg4cOHDL9Xx9fSX9dfmgdHMiNcBeuKYbAIAC7uuvv9Zvv/2mxx57TEWLFtXq1auVnp5uPQUzKChIP//8sxITE1WoUCEVK1bMzhUDwO0VLlxYXbp00ZtvvqlixYqpePHiGj58uBwcHKxnJGalfPnyCgwM1IgRI/TOO+/owIEDmjx5cj5WDthipBsAgAKuSJEiWrlypRo3bqyQkBDNnTtXS5cu1cMPPyxJ6t+/vxwdHVWpUiX5+voqKSnJzhUDQM5MmTJFderU0VNPPaWmTZuqXr16CgkJsU5ymxVnZ2ctXbpUv/76q6pWrarx48drzJgx+Vg1YMti/P1iBwAAAAC4RyUnJ6tkyZKaPHmyunfvbu9ygBzh9HIAAAAA96Rdu3bp119/Va1atXThwgWNGjVKktS6dWs7VwbkHKEbAAAAwD1r0qRJ2r9/v1xcXBQWFqYNGzbIx8fH3mUBOcbp5QAAAAAAmISJ1AAAAAAAMAmhGwAAAAAAkxC6AQAAAAAwCaEbAAAAAACTELoBAAAAADAJoRsAgAIsNjZWFotF58+fz/E6QUFBmjZtmmk1AQCAvxC6AQAwSdeuXWWxWNSrV69My3r37i2LxaKuXbvmf2G3EBQUJIvFku3jXqsXAIB7nZO9CwAA4H4WGBioZcuWaerUqXJ3d5ckXbt2TdHR0SpVqpSdq8ts27ZtSktLkyRt2rRJbdu21f79++Xl5SVJ1mMAAAA5w0g3AAAmqlGjhgIDA7Vy5Upr28qVK1WqVClVr17dpm9KSor69Omj4sWLy83NTfXr19e2bdts+qxevVoVKlSQu7u7Hn/8cSUmJmba508//aQGDRrI3d1dgYGB6tOnj5KTk3NUr6+vr/z9/eXv769ixYpJkooXLy4/Pz/Vr19f8+fPt+kfFxcni8WiQ4cOSZIsFovmzJmj5s2by93dXWXLltVnn31ms87Ro0fVrl07FSlSRMWKFVPr1q2zPA4AAO4HhG4AAEz20ksvaeHChdbnCxYsULdu3TL1GzBggFasWKHFixdr586dKl++vMLDw3Xu3DlJN8Pqs88+q1atWikuLk49evTQoEGDbLaRkJCgZs2aqW3bttqzZ4+WL1+un376SZGRkXd0DBaLJdNxSNLChQv12GOPqXz58ta2oUOHqm3bttq9e7c6deqkF154QfHx8ZKk69evKzw8XIULF9aGDRu0ceNGFSpUSM2aNVNqauod1QgAwL2I0A0AgMlefPFF/fTTTzpy5IiOHDmijRs36sUXX7Tpk5ycrDlz5mjixIlq3ry5KlWqpPnz58vd3V0ffPCBJGnOnDkqV66cJk+erIoVK6pTp06ZrrEeO3asOnXqpL59+yo4OFh169bVjBkz9OGHH+ratWt3dBxdu3bV/v37tXXrVkk3A3R0dLReeuklm37PP/+8evTooQoVKmj06NGqWbOmZs6cKUlavny50tPT9f7776tKlSoKCQnRwoULlZSUpNjY2DuqDwCAexHXdAMAYDJfX1+1bNlSixYtkmEYatmypXx8fGz6JCQk6Pr166pXr561zdnZWbVq1bKOEsfHx6t27do269WpU8fm+e7du7Vnzx4tWbLE2mYYhtLT03X48GGFhITk+TgCAgLUsmVLLViwQLVq1dJ///tfpaSk6Pnnn79lTXXq1FFcXJy1vkOHDqlw4cI2fa5du6aEhIQ81wYAwL2K0A0AQD546aWXrKd4z54927T9XL58Wa+88or69OmTadndmLitR48e6ty5s6ZOnaqFCxeqffv28vDwyFV9YWFhNn8UyODr63vH9QEAcK8hdAMAkA8yrlm2WCwKDw/PtLxcuXJycXHRxo0bVbp0aUk3T9/etm2b+vbtK0kKCQnRV199ZbPeli1bbJ7XqFFD+/bts7nG+m5q0aKFPD09NWfOHK1Zs0Y//vhjpj5btmxRRESEzfOMSeNq1Kih5cuXq3jx4tYZ0QEAuJ9xTTcAAPnA0dFR8fHx2rdvnxwdHTMt9/T01Kuvvqo333xTa9as0b59+9SzZ09duXJF3bt3lyT16tVLBw8e1Jtvvqn9+/crOjpaixYtstnOwIEDtWnTJkVGRiouLk4HDx7Ul19+eccTqf39OLp27arBgwcrODg406nkkvTpp59qwYIFOnDggIYPH66tW7da99+pUyf5+PiodevW2rBhgw4fPqzY2Fj16dNHv//++12pEQCAewmhGwCAfOLl5XXL0d1x48apbdu26ty5s2rUqKFDhw5p7dq1Klq0qKSbp4evWLFCX3zxhapVq6a5c+fq3XfftdlG1apV9cMPP+jAgQNq0KCBqlevrmHDhikgIOCuHUf37t2Vmpqa5QzskjRy5EgtW7ZMVatW1YcffqilS5eqUqVKkiQPDw/9+OOPKlWqlJ599lmFhISoe/fuunbtGiPfAID7ksUwDMPeRQAAgIJjw4YNatKkiY4ePSo/Pz+bZRaLRZ9//rnatGljn+IAALjHcE03AADIkZSUFJ0+fVojRozQ888/nylwAwCAzDi9HAAA5MjSpUtVunRpnT9/XhMmTLB3OQAAFAicXg4AAAAAgEkY6QYAAAAAwCSEbgAAAAAATELoBgAAAADAJIRuAAAAAABMQugGAAAAAMAkhG4AAAAAAExC6AYAAAAAwCSEbgAAAAAATELoBgAAAADAJP8PwbdVbvn1l3kAAAAASUVORK5CYII=\n"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Conclusion\n",
        "\n",
        "Congratulations on completing this comprehensive lab on Recurrent Neural Networks (RNNs) for sentiment analysis using the IMDB dataset! Throughout this exercise, you've successfully:\n",
        "\n",
        "1. Loaded and preprocessed the IMDB dataset using the `datasets` library and PyTorch\n",
        "2. Implemented a custom `IMDBDataset` class for efficient data handling\n",
        "3. Designed and implemented a flexible `TextClassifier` model supporting three RNN architectures:\n",
        "   - Simple RNN\n",
        "   - Long Short-Term Memory (LSTM)\n",
        "   - Gated Recurrent Unit (GRU)\n",
        "4. Set up the training process with CrossEntropyLoss and Adam optimizer\n",
        "5. Trained and evaluated each RNN architecture on the IMDB dataset\n",
        "6. Evaluated the models' performance using various metrics:\n",
        "   - Accuracy\n",
        "   - Precision\n",
        "   - Recall\n",
        "   - F1 Score\n",
        "   - Loss\n",
        "7. Visualized the results using a grouped bar chart to compare model performances\n",
        "\n",
        "Key takeaways from this lab:\n",
        "\n",
        "- Understanding the structure and implementation of different RNN architectures in PyTorch\n",
        "- The importance of proper data preprocessing and tokenization for NLP tasks\n",
        "- The process of training and evaluating multiple model architectures on the same dataset\n",
        "- The significance of various evaluation metrics in assessing model performance for sentiment analysis\n",
        "- The value of visualizations in comparing the performance of different model architectures\n",
        "\n",
        "This lab has provided you with hands-on experience in building, training, and evaluating RNN-based models for text classification. You've gained insights into the strengths and weaknesses of different RNN architectures and their application to sentiment analysis. These skills form a solid foundation for tackling more complex natural language processing tasks and understanding advanced sequence modeling techniques."
      ],
      "metadata": {
        "id": "JfrtE41Zi_9R"
      }
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "xrZxHMCEjASD"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}