Light_04_python_rnn_findespace_rnn.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "deep_env",
      "language": "python",
      "name": "deep_env"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.4"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/hocine-drioueche/OpenclassroomsProject/blob/main/Light_04_python_rnn_findespace_rnn.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hu7mQQtYFs0Q"
      },
      "source": [
        "<img src=\"https://datascientest.fr/train/assets/logo_datascientest.png\" style=\"height:150px\">\n",
        "\n",
        "<hr style=\"border-width:2px;border-color:#75DFC1\">\n",
        "<h1 style = \"text-align:center\" > Réseau de neurones récurrents </h1>\n",
        "<h2 style = \"text-align:center\" > Trouver les espaces dans une phrase </h2>\n",
        "<hr style=\"border-width:2px;border-color:#75DFC1\">\n",
        "\n",
        "Ce notebook est destiné à pratiquer les notions évoquées dans le quatrième exercice du module sur une machine plus adaptée.\n",
        "\n",
        "Si c'est la première fois que vous utilisez colab, n'hésitez pas à jeter un coup d'oeil sur ce [notebook](https://colab.research.google.com/drive/1jXEKOk3mRYBqFWoVwJ0ZpsRJCWj46Yxt?usp=sharing).\n",
        "\n",
        "\n",
        "> Dans cet exercice, vous allez vous exercer à utiliser un modèle RNN pour trouver les espaces d'une phrase. En d'autres termes, vous allez trouver l'emplacement des espaces dans une phrase où on les a préalable enlevé:\n",
        ">\n",
        "><b>Input</b> : <i>\"Iwasawillingworkerandstoodwellwiththeguards.\"</i>\n",
        ">\n",
        "><b>Output</b> : <i>\"I was a willing worker and stood well with the guards.\"</i>\n",
        "\n",
        "\n",
        "## Modélisation du problème\n",
        "\n",
        "> Dans cet exercice, nous allons résoudre notre problème en prédisant pour chaque caractère s'il est précédé ou non par un espace.\n",
        "## Jeu de données\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bEIf-obrFs0T",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "34aa4d59-aa46-47f5-b763-3a1ee35184c8"
      },
      "source": [
        "## Importation des données et packages\n",
        "!pip install swifter\n",
        "!python3 -m nltk.downloader stopwords\n",
        "!python3 -m nltk.downloader punkt\n",
        "!wget https://train-exo.s3.eu-west-1.amazonaws.com/677/MovieReview.csv\n",
        "!wget https://train-exo.s3.eu-west-1.amazonaws.com/678/findspace.h5\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Collecting swifter\n",
            "  Downloading swifter-1.3.3.tar.gz (821 kB)\n",
            "\u001b[K     |████████████████████████████████| 821 kB 36.8 MB/s \n",
            "\u001b[?25hRequirement already satisfied: pandas>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from swifter) (1.3.5)\n",
            "Collecting psutil>=5.6.6\n",
            "  Downloading psutil-5.9.1-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (281 kB)\n",
            "\u001b[K     |████████████████████████████████| 281 kB 71.9 MB/s \n",
            "\u001b[?25hRequirement already satisfied: dask[dataframe]>=2.10.0 in /usr/local/lib/python3.7/dist-packages (from swifter) (2.12.0)\n",
            "Requirement already satisfied: tqdm>=4.33.0 in /usr/local/lib/python3.7/dist-packages (from swifter) (4.64.0)\n",
            "Requirement already satisfied: ipywidgets>=7.0.0 in /usr/local/lib/python3.7/dist-packages (from swifter) (7.7.1)\n",
            "Requirement already satisfied: cloudpickle>=0.2.2 in /usr/local/lib/python3.7/dist-packages (from swifter) (1.3.0)\n",
            "Requirement already satisfied: parso>0.4.0 in /usr/local/lib/python3.7/dist-packages (from swifter) (0.8.3)\n",
            "Requirement already satisfied: bleach>=3.1.1 in /usr/local/lib/python3.7/dist-packages (from swifter) (5.0.1)\n",
            "Requirement already satisfied: six>=1.9.0 in /usr/local/lib/python3.7/dist-packages (from bleach>=3.1.1->swifter) (1.15.0)\n",
            "Requirement already satisfied: webencodings in /usr/local/lib/python3.7/dist-packages (from bleach>=3.1.1->swifter) (0.5.1)\n",
            "Collecting partd>=0.3.10\n",
            "  Downloading partd-1.2.0-py3-none-any.whl (19 kB)\n",
            "Requirement already satisfied: numpy>=1.13.0 in /usr/local/lib/python3.7/dist-packages (from dask[dataframe]>=2.10.0->swifter) (1.21.6)\n",
            "Requirement already satisfied: toolz>=0.7.3 in /usr/local/lib/python3.7/dist-packages (from dask[dataframe]>=2.10.0->swifter) (0.12.0)\n",
            "Collecting fsspec>=0.6.0\n",
            "  Downloading fsspec-2022.7.1-py3-none-any.whl (141 kB)\n",
            "\u001b[K     |████████████████████████████████| 141 kB 67.9 MB/s \n",
            "\u001b[?25hRequirement already satisfied: ipykernel>=4.5.1 in /usr/local/lib/python3.7/dist-packages (from ipywidgets>=7.0.0->swifter) (4.10.1)\n",
            "Requirement already satisfied: jupyterlab-widgets>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from ipywidgets>=7.0.0->swifter) (1.1.1)\n",
            "Requirement already satisfied: ipython-genutils~=0.2.0 in /usr/local/lib/python3.7/dist-packages (from ipywidgets>=7.0.0->swifter) (0.2.0)\n",
            "Requirement already satisfied: widgetsnbextension~=3.6.0 in /usr/local/lib/python3.7/dist-packages (from ipywidgets>=7.0.0->swifter) (3.6.1)\n",
            "Requirement already satisfied: traitlets>=4.3.1 in /usr/local/lib/python3.7/dist-packages (from ipywidgets>=7.0.0->swifter) (5.1.1)\n",
            "Requirement already satisfied: ipython>=4.0.0 in /usr/local/lib/python3.7/dist-packages (from ipywidgets>=7.0.0->swifter) (5.5.0)\n",
            "Requirement already satisfied: tornado>=4.0 in /usr/local/lib/python3.7/dist-packages (from ipykernel>=4.5.1->ipywidgets>=7.0.0->swifter) (5.1.1)\n",
            "Requirement already satisfied: jupyter-client in /usr/local/lib/python3.7/dist-packages (from ipykernel>=4.5.1->ipywidgets>=7.0.0->swifter) (5.3.5)\n",
            "Requirement already satisfied: simplegeneric>0.8 in /usr/local/lib/python3.7/dist-packages (from ipython>=4.0.0->ipywidgets>=7.0.0->swifter) (0.8.1)\n",
            "Requirement already satisfied: decorator in /usr/local/lib/python3.7/dist-packages (from ipython>=4.0.0->ipywidgets>=7.0.0->swifter) (4.4.2)\n",
            "Requirement already satisfied: prompt-toolkit<2.0.0,>=1.0.4 in /usr/local/lib/python3.7/dist-packages (from ipython>=4.0.0->ipywidgets>=7.0.0->swifter) (1.0.18)\n",
            "Requirement already satisfied: pexpect in /usr/local/lib/python3.7/dist-packages (from ipython>=4.0.0->ipywidgets>=7.0.0->swifter) (4.8.0)\n",
            "Requirement already satisfied: pickleshare in /usr/local/lib/python3.7/dist-packages (from ipython>=4.0.0->ipywidgets>=7.0.0->swifter) (0.7.5)\n",
            "Requirement already satisfied: pygments in /usr/local/lib/python3.7/dist-packages (from ipython>=4.0.0->ipywidgets>=7.0.0->swifter) (2.6.1)\n",
            "Requirement already satisfied: setuptools>=18.5 in /usr/local/lib/python3.7/dist-packages (from ipython>=4.0.0->ipywidgets>=7.0.0->swifter) (57.4.0)\n",
            "Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.7/dist-packages (from pandas>=1.0.0->swifter) (2.8.2)\n",
            "Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.7/dist-packages (from pandas>=1.0.0->swifter) (2022.1)\n",
            "Collecting locket\n",
            "  Downloading locket-1.0.0-py2.py3-none-any.whl (4.4 kB)\n",
            "Requirement already satisfied: wcwidth in /usr/local/lib/python3.7/dist-packages (from prompt-toolkit<2.0.0,>=1.0.4->ipython>=4.0.0->ipywidgets>=7.0.0->swifter) (0.2.5)\n",
            "Requirement already satisfied: notebook>=4.4.1 in /usr/local/lib/python3.7/dist-packages (from widgetsnbextension~=3.6.0->ipywidgets>=7.0.0->swifter) (5.3.1)\n",
            "Requirement already satisfied: jinja2 in /usr/local/lib/python3.7/dist-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets>=7.0.0->swifter) (2.11.3)\n",
            "Requirement already satisfied: nbconvert in /usr/local/lib/python3.7/dist-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets>=7.0.0->swifter) (5.6.1)\n",
            "Requirement already satisfied: jupyter-core>=4.4.0 in /usr/local/lib/python3.7/dist-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets>=7.0.0->swifter) (4.11.1)\n",
            "Requirement already satisfied: Send2Trash in /usr/local/lib/python3.7/dist-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets>=7.0.0->swifter) (1.8.0)\n",
            "Requirement already satisfied: nbformat in /usr/local/lib/python3.7/dist-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets>=7.0.0->swifter) (5.4.0)\n",
            "Requirement already satisfied: terminado>=0.8.1 in /usr/local/lib/python3.7/dist-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets>=7.0.0->swifter) (0.13.3)\n",
            "Requirement already satisfied: pyzmq>=13 in /usr/local/lib/python3.7/dist-packages (from jupyter-client->ipykernel>=4.5.1->ipywidgets>=7.0.0->swifter) (23.2.0)\n",
            "Requirement already satisfied: ptyprocess in /usr/local/lib/python3.7/dist-packages (from terminado>=0.8.1->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets>=7.0.0->swifter) (0.7.0)\n",
            "Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.7/dist-packages (from jinja2->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets>=7.0.0->swifter) (2.0.1)\n",
            "Requirement already satisfied: defusedxml in /usr/local/lib/python3.7/dist-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets>=7.0.0->swifter) (0.7.1)\n",
            "Requirement already satisfied: entrypoints>=0.2.2 in /usr/local/lib/python3.7/dist-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets>=7.0.0->swifter) (0.4)\n",
            "Requirement already satisfied: mistune<2,>=0.8.1 in /usr/local/lib/python3.7/dist-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets>=7.0.0->swifter) (0.8.4)\n",
            "Requirement already satisfied: pandocfilters>=1.4.1 in /usr/local/lib/python3.7/dist-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets>=7.0.0->swifter) (1.5.0)\n",
            "Requirement already satisfied: testpath in /usr/local/lib/python3.7/dist-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets>=7.0.0->swifter) (0.6.0)\n",
            "Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.7/dist-packages (from nbformat->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets>=7.0.0->swifter) (4.3.3)\n",
            "Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.7/dist-packages (from nbformat->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets>=7.0.0->swifter) (2.16.1)\n",
            "Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.7/dist-packages (from jsonschema>=2.6->nbformat->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets>=7.0.0->swifter) (4.12.0)\n",
            "Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.7/dist-packages (from jsonschema>=2.6->nbformat->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets>=7.0.0->swifter) (0.18.1)\n",
            "Requirement already satisfied: importlib-resources>=1.4.0 in /usr/local/lib/python3.7/dist-packages (from jsonschema>=2.6->nbformat->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets>=7.0.0->swifter) (5.9.0)\n",
            "Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.7/dist-packages (from jsonschema>=2.6->nbformat->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets>=7.0.0->swifter) (22.1.0)\n",
            "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from jsonschema>=2.6->nbformat->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets>=7.0.0->swifter) (4.1.1)\n",
            "Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.7/dist-packages (from importlib-resources>=1.4.0->jsonschema>=2.6->nbformat->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets>=7.0.0->swifter) (3.8.1)\n",
            "Building wheels for collected packages: swifter\n",
            "  Building wheel for swifter (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for swifter: filename=swifter-1.3.3-py3-none-any.whl size=16253 sha256=1e0f46dfdb5208abd50f26388074e266cc4d0e032eef08dbfaf6d10b33faab14\n",
            "  Stored in directory: /root/.cache/pip/wheels/64/bf/da/da0022edab5fd84114858a95e4f32f2fca0d5b7d758905f594\n",
            "Successfully built swifter\n",
            "Installing collected packages: locket, partd, fsspec, psutil, swifter\n",
            "  Attempting uninstall: psutil\n",
            "    Found existing installation: psutil 5.4.8\n",
            "    Uninstalling psutil-5.4.8:\n",
            "      Successfully uninstalled psutil-5.4.8\n",
            "Successfully installed fsspec-2022.7.1 locket-1.0.0 partd-1.2.0 psutil-5.9.1 swifter-1.3.3\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.colab-display-data+json": {
              "pip_warning": {
                "packages": [
                  "psutil"
                ]
              }
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "/usr/lib/python3.7/runpy.py:125: RuntimeWarning: 'nltk.downloader' found in sys.modules after import of package 'nltk', but prior to execution of 'nltk.downloader'; this may result in unpredictable behaviour\n",
            "  warn(RuntimeWarning(msg))\n",
            "[nltk_data] Downloading package stopwords to /root/nltk_data...\n",
            "[nltk_data]   Unzipping corpora/stopwords.zip.\n",
            "/usr/lib/python3.7/runpy.py:125: RuntimeWarning: 'nltk.downloader' found in sys.modules after import of package 'nltk', but prior to execution of 'nltk.downloader'; this may result in unpredictable behaviour\n",
            "  warn(RuntimeWarning(msg))\n",
            "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
            "[nltk_data]   Unzipping tokenizers/punkt.zip.\n",
            "--2022-08-04 13:18:45--  https://train-exo.s3.eu-west-1.amazonaws.com/677/MovieReview.csv\n",
            "Resolving train-exo.s3.eu-west-1.amazonaws.com (train-exo.s3.eu-west-1.amazonaws.com)... 52.218.37.128\n",
            "Connecting to train-exo.s3.eu-west-1.amazonaws.com (train-exo.s3.eu-west-1.amazonaws.com)|52.218.37.128|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 32950157 (31M) [text/csv]\n",
            "Saving to: ‘MovieReview.csv’\n",
            "\n",
            "MovieReview.csv     100%[===================>]  31.42M  8.39MB/s    in 3.7s    \n",
            "\n",
            "2022-08-04 13:18:49 (8.39 MB/s) - ‘MovieReview.csv’ saved [32950157/32950157]\n",
            "\n",
            "--2022-08-04 13:18:49--  https://train-exo.s3.eu-west-1.amazonaws.com/678/findspace.h5\n",
            "Resolving train-exo.s3.eu-west-1.amazonaws.com (train-exo.s3.eu-west-1.amazonaws.com)... 52.218.37.128\n",
            "Connecting to train-exo.s3.eu-west-1.amazonaws.com (train-exo.s3.eu-west-1.amazonaws.com)|52.218.37.128|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 1803544 (1.7M) [binary/octet-stream]\n",
            "Saving to: ‘findspace.h5’\n",
            "\n",
            "findspace.h5        100%[===================>]   1.72M  1.35MB/s    in 1.3s    \n",
            "\n",
            "2022-08-04 13:18:52 (1.35 MB/s) - ‘findspace.h5’ saved [1803544/1803544]\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r7-pSQ1xaNAi"
      },
      "source": [
        "Un des interêts principaux de colab est la mise à disposition d'un GPU. Utiliser un GPU permet d'accelerer grandement l'execution et donc l'entrainement de modèle de deep learning. Pour configurer le GPU (processeur graphique), il suffit de cliquer sur Edit > Notebook settings et sélectionner GPU comme accélérateur matériel.\n",
        "\n",
        "* Exécuter la cellule suivante pour vérifier que le GPU soit bien activé."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zDivBVVaaPyN",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "fde1a692-9c54-4458-9bc7-37c1e40b9f9d"
      },
      "source": [
        "import tensorflow as tf\n",
        "if tf.test.gpu_device_name():\n",
        "    print('Default GPU Device:{}'.format(tf.test.gpu_device_name()))\n",
        "else:\n",
        "    print(\"Please change your hardware accelerator\")"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Default GPU Device:/device:GPU:0\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3z0XOrz8H4KL"
      },
      "source": [
        "\n",
        "* Importer le module **pandas** sous le nom **pd**\n",
        "\n",
        "\n",
        "* Charger le fichier ***'MovieReview.csv'*** sous le nom **df**.\n",
        "\n",
        "\n",
        "* Afficher les 5 premières lignes du dataframe."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "M1A82ukqIu-2"
      },
      "source": [
        "## Insérez votre code ici"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cBaiBu4RFs0U",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "outputId": "43f1f89e-0023-40e6-bbee-399c1df4b258"
      },
      "source": [
        "#@title Solution\n",
        "import pandas as pd\n",
        "df = pd.read_csv('MovieReview.csv')\n",
        "df.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "  sentiment                                             review\n",
              "0  Positive  With all this stuff going down at the moment w...\n",
              "1  Positive  'The Classic War of the Worlds' by Timothy Hin...\n",
              "2  Negative  The film starts with a manager (Nicholas Bell)...\n",
              "3  Negative  It must be assumed that those who praised this...\n",
              "4  Positive  Superbly trashy and wondrously unpretentious 8..."
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-cd5c72ed-25a2-4866-be05-0f39a693c935\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>sentiment</th>\n",
              "      <th>review</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Positive</td>\n",
              "      <td>With all this stuff going down at the moment w...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Positive</td>\n",
              "      <td>'The Classic War of the Worlds' by Timothy Hin...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Negative</td>\n",
              "      <td>The film starts with a manager (Nicholas Bell)...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Negative</td>\n",
              "      <td>It must be assumed that those who praised this...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Positive</td>\n",
              "      <td>Superbly trashy and wondrously unpretentious 8...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-cd5c72ed-25a2-4866-be05-0f39a693c935')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-cd5c72ed-25a2-4866-be05-0f39a693c935 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-cd5c72ed-25a2-4866-be05-0f39a693c935');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_qcikxDtFs0V"
      },
      "source": [
        "## Preprocessing\n",
        "\n",
        "* Appliquer la fonction de preprocessing `preprocess_sentence` à **df.review** pour nettoyer les reviews."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pDNF3YnNFs0V",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "outputId": "82fccfe9-c830-4a0b-d359-0484eabf0e1b"
      },
      "source": [
        "import re, unicodedata\n",
        "\n",
        "# Converts the unicode file to ascii\n",
        "def unicode_to_ascii(s):\n",
        "    return ''.join(c for c in unicodedata.normalize('NFD', s)\n",
        "        if unicodedata.category(c) != 'Mn')\n",
        "\n",
        "def preprocess_sentence(w):\n",
        "    w = unicode_to_ascii(w.lower().strip())\n",
        "    w = re.sub(r\"[^a-zA-Z?.!,]+\", \" \", w)\n",
        "    w = re.sub(r'\\b\\w{0,1}\\b', '', w)\n",
        "    w = re.sub(r'\\ +', \" \", w)\n",
        "    return w\n",
        "\n",
        "df.review = df.review.apply(lambda x :preprocess_sentence(x))\n",
        "df.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "  sentiment                                             review\n",
              "0  Positive  with all this stuff going down at the moment w...\n",
              "1  Positive   the classic war of the worlds by timothy hine...\n",
              "2  Negative  the film starts with manager nicholas bell giv...\n",
              "3  Negative  it must be assumed that those who praised this...\n",
              "4  Positive  superbly trashy and wondrously unpretentious e..."
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-af5a4c63-a798-4a7c-92d2-11bece828dee\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>sentiment</th>\n",
              "      <th>review</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Positive</td>\n",
              "      <td>with all this stuff going down at the moment w...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Positive</td>\n",
              "      <td>the classic war of the worlds by timothy hine...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Negative</td>\n",
              "      <td>the film starts with manager nicholas bell giv...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Negative</td>\n",
              "      <td>it must be assumed that those who praised this...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Positive</td>\n",
              "      <td>superbly trashy and wondrously unpretentious e...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-af5a4c63-a798-4a7c-92d2-11bece828dee')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-af5a4c63-a798-4a7c-92d2-11bece828dee button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-af5a4c63-a798-4a7c-92d2-11bece828dee');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZoREF5W2Fs0W"
      },
      "source": [
        "* Stocker dans une variable **text** l'ensemble du texte sous forme d'une chaîne de caractère."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_1ZT7i47Fs0W"
      },
      "source": [
        "## Insérez votre code ici\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rh-YWb3sFs0W",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "84b269c4-ba34-4c98-ce60-e108e1adbb88"
      },
      "source": [
        "#@title Solution\n",
        "from tqdm import tqdm\n",
        "text = ''\n",
        "for review in tqdm(df.review):\n",
        "    text += ' ' + review"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 25000/25000 [00:00<00:00, 290234.33it/s]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EALQ5LJ9Fs0X"
      },
      "source": [
        "* Exécuter la cellule suivante pour définir les variables du vocabulaire."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LJcpIEQoFs0X",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "153fec76-f36d-4e56-d774-cbd0913c63ed"
      },
      "source": [
        "import numpy as np\n",
        "vocab = sorted(list(set(text)))\n",
        "vocab[0]=''\n",
        "print('{} uniques characters'.format(len(vocab)))\n",
        "print(vocab)\n",
        "char2idx = {u:i for i, u in enumerate(vocab)}\n",
        "idx2char = np.array(vocab)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "31 uniques characters\n",
            "['', '!', ',', '.', '?', 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z']\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eTkxy366Fs0X"
      },
      "source": [
        "* Extraire l'ensemble des phrases dans la variable **`sentences`** à l'aide de la fonction `sent_tokenize` de **`nltk.tokenize`**. La fonction `sent_tokenize` permet de découper un texte en une liste de phrases.\n",
        "\n",
        "\n",
        "* Afficher la distribution du nombre de lettre de chaque phrase."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "j-FOtoZZFs0Y"
      },
      "source": [
        "## Insérez votre code ici\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LCVfdVsFFs0Y"
      },
      "source": [
        "#@title Solution\n",
        "from nltk.tokenize import sent_tokenize\n",
        "import seaborn as sns\n",
        "sentences = np.array(sent_tokenize(text))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9rdSuEvMFs0Z"
      },
      "source": [
        "* Pour éviter des séquences trop long ou trop court, stocker dans la variable **sentences** les phrases entre 50 et 200 caractères."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4JZFWtYTFs0Z"
      },
      "source": [
        "## Insérez votre code ici\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8hfR1JFwFs0Z",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a6549508-6bdb-448f-8112-1736a349d41c"
      },
      "source": [
        "#@title Solution\n",
        "sentences = np.array(sentences)\n",
        "seuil = 50\n",
        "seuil2 = 200\n",
        "sentences = sentences[np.array(list(map(len, sentences)))>seuil]\n",
        "sentences = sentences[np.array(list(map(len, sentences)))<seuil2]\n",
        "len(sentences)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "207178"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8DFqPJ3HFs0Z"
      },
      "source": [
        "* Définir une fonction `conv2input` enlevant tous les espaces d'une liste de phrase.\n",
        "\n",
        "\n",
        "* Tester la fonction sur la liste de phrase suivante :\n",
        "[<i>'stories like this with this depth and feeling and this intricacy of meaning are very rare.'</i>,\n",
        "<i>'it is sad story but i ve never encountered any catharsis more beautifully made.'</i>]"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "J9FpIpBoFs0a"
      },
      "source": [
        "## Insérez votre code ici\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Pos_65MUFs0a",
        "scrolled": true
      },
      "source": [
        "#@title Solution\n",
        "def conv2input(sentences):\n",
        "    targets = []\n",
        "    for s in sentences:\n",
        "        targets.append(re.sub(r\" \", \"\", s))\n",
        "        # Or\n",
        "        # targets.append(s.replace(\" \", ''))\n",
        "    return targets\n",
        "\n",
        "conv2input(['stories like this with this depth and feeling and this intricacy of meaning are very rare.',\n",
        "'it is sad story but i ve never encountered any catharsis more beautifully made.'])"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "U94k6w0QFs0a"
      },
      "source": [
        "* Convertir l'ensemble des phrases **sentences** dans une variable **inputs** à l'aide de la fonction `conv2target`."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WLXL5WIUFs0a"
      },
      "source": [
        "## Insérez votre code ici\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BkdrNOrZFs0b"
      },
      "source": [
        "#@title Solution\n",
        "inputs = conv2input(sentences)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4sexJp6SFs0b"
      },
      "source": [
        "* Exécuter la cellule pour convertir toutes phrases de **`inputs`** en une liste d'index de caractère. Le résultat sera stocké dans les variables **`sentences_inputs`**, **`max_length_inputs`**."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6Jyud3j6Fs0c"
      },
      "source": [
        "def max_length(sentences):\n",
        "    return max(map(len, sentences))\n",
        "\n",
        "def process_data(sentences):\n",
        "    max_len = max_length(sentences)\n",
        "    text_as_int = np.zeros([len(sentences), max_len], dtype=int)\n",
        "    for i, s in enumerate(sentences):\n",
        "        for j, c in enumerate(s):\n",
        "            text_as_int[i,j] = char2idx[c]\n",
        "    return text_as_int, max_len\n",
        "\n",
        "\n",
        "sentences_inputs, max_length_inputs = process_data(inputs)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "H30sjderFs0c"
      },
      "source": [
        ">\n",
        "> Puisque dans la partie précédente, nous avons ajouté des 0 pour toutes les séquences aient la même forme, il est nécessaire de définir une troisième classe qui va faire effet de masque sur nos sorties.\n",
        "\n",
        "* Exécuter la cellule suivante pour définir la variable **`sentences_targets`**."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DbQL5-1JFs0c"
      },
      "source": [
        "def process_target(sentences, max_len):\n",
        "    text_as_int = np.zeros([len(sentences), max_len], dtype=int)\n",
        "    for i, s in enumerate(sentences):\n",
        "        j = -1\n",
        "        for p in s.split(' ')[:-1]:\n",
        "            j += len(p)\n",
        "            text_as_int[i, j]=1\n",
        "        j += len(s.split(' ')[-1])\n",
        "        text_as_int[i,j:]=2\n",
        "    return text_as_int\n",
        "\n",
        "sentences_targets = process_target(sentences, max_length_inputs)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "J9JSsbgWFs0c"
      },
      "source": [
        "> Maintenant que les données sont prétraités et mises en forme, il est nécessaire de les séparer en un ensemble d'entraînement et de validation.\n",
        "\n",
        "* Créer les dataframes **X_train**, **X_test**, **y_train**, **y_test** à partir de **sentences_inputs** et **sentences_targets** en utilisant la fonction `train_test_split` et en gardant 20% des données pour l'échantillon de test. Par souci de reproductibilité des résultats, utiliser random_state=1234."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xflZL-JSFs0d"
      },
      "source": [
        "## Insérez votre code ici\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xRrQs0q9Fs0d"
      },
      "source": [
        "#@title Solution\n",
        "from sklearn.model_selection import train_test_split\n",
        "# Creating training and validation sets using an 80-20 split\n",
        "X_train, X_test, y_train, y_test = train_test_split(sentences_inputs, sentences_targets, test_size=0.2, random_state=1234)\n",
        "\n",
        "# Show length\n",
        "print('Shape of sentence X :', X_train.shape)\n",
        "print('Shape of no space sentence y :', y_train.shape)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "C5PvSrxKFs0e"
      },
      "source": [
        "## Modélisation\n",
        "\n",
        "Nous allons implementer un **Bidirectionnal RNN**.\n",
        "\n",
        "* Définir un modèle `Sequential` sous le nom **`model`**.\n",
        "\n",
        "\n",
        "* Ajouter une couche `Embedding` en précisant une entrée de taille **`len(vocab)`** et une sortie de taille **`len(vocab)`**.\n",
        "\n",
        "\n",
        "* Ajouter une couche `Bidirectional` composé de `GRU` avec 256 neurones et retournant une séquence (argument **`return_sequences`** de la fonction `GRU`).\n",
        "\n",
        "\n",
        "* Ajouter une couche `Dropout` afin limiter le sur-apprentissage.\n",
        "\n",
        "\n",
        "* Ajouter une dernière couche `Dense` de 3 neurones et une fonction d'activation `softmax`.\n",
        "\n",
        "\n",
        "* Afficher le résumé du modèle."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cHWJbh2LFs0e"
      },
      "source": [
        "## Insérez votre code ici\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jYMe7ajtFs0e"
      },
      "source": [
        "#@title Solution\n",
        "from tensorflow.keras import Sequential\n",
        "from tensorflow.keras.layers import Dense, RNN, GRUCell, Conv2D, Embedding, Bidirectional, Dropout\n",
        "\n",
        "model = Sequential()\n",
        "model.add(Embedding(len(vocab), len(vocab)))\n",
        "model.add(Bidirectional(RNN(GRUCell(256, recurrent_initializer='glorot_uniform'),\n",
        "                                return_sequences=True)))\n",
        "model.add(Dropout(0.3))\n",
        "model.add(Dense(3, activation='softmax'))\n",
        "model.summary()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mN7IDB8CFs0f"
      },
      "source": [
        "\n",
        "> Comme l'apprentissage d'un algorithme de deep learning est lié à sa fonction perte, nous allons appliquer un masque sur toutes les valeurs de fonction de perte qui ont un label de valeur 2 :\n",
        ">\n",
        "> <img src='https://assets-datascientest.s3-eu-west-1.amazonaws.com/notebooks/python_deepnlp_findspace_mask.png' style='width:800px'>\n",
        ">\n",
        "> Comme nous sommes face à un problème de classification, nous allons **appliquer notre masque à la fonction de perte sparseCategoricalCrossentropy**.\n",
        "\n",
        "* Exécuter la cellule suivante pour définir la fonction de perte `loss_function`."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cvaUFP8TFs0f"
      },
      "source": [
        "import tensorflow as tf\n",
        "\n",
        "loss_object = tf.keras.losses.SparseCategoricalCrossentropy()\n",
        "\n",
        "def loss_function(real, pred):\n",
        "    # Mask\n",
        "    mask = tf.math.logical_not(tf.math.equal(real, 2))\n",
        "    # Avoid type error\n",
        "    mask = tf.cast(mask, dtype=pred.dtype)\n",
        "    # Loss function\n",
        "    loss_ = loss_object(real, pred)\n",
        "    # Apply mask on loss function\n",
        "    loss_ *= mask\n",
        "    return tf.reduce_mean(loss_)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "j_HymKjXFs0f"
      },
      "source": [
        "* Compiler le modèle avec la fonction de perte `loss_function` et un optimizer **'adam'**."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "K7vaUwgrFs0f"
      },
      "source": [
        "## Insérez votre code ici\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8jSqZUnaFs0g"
      },
      "source": [
        "#@title Solution\n",
        "model.compile(loss=loss_function, optimizer='adam')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NFtTCcQOFs0g"
      },
      "source": [
        "* Pour une raison de temps de calcul, nous avons déjà entraîner le modèle. Charger les poids **'findspace.h2'** à l'aide de la méthode `load_weights`."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MEcfRx02Fs0g"
      },
      "source": [
        "## Insérez votre code ici\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XQQvvxlDFs0g"
      },
      "source": [
        "#@title Solution\n",
        "#model.fit(X_train, y_train, batch_size=32, epochs=1)\n",
        "model.load_weights('findspace.h5')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Rwe94hV2Fs0g"
      },
      "source": [
        "* Prédire la probabilité de chaque élément de la séquence **X_test[:100]** à l'aide de la méthonde `predict`. Nous prenons que les 100 premiers éléments de **X_test** pour une raison de temps de calcul. Stocker le résultat dans un tableau nommé **y_prob**.\n",
        "\n",
        "\n",
        "* Appliquer la méthode `argmax` du tableau **y_prob** pour obtenir les classes les plus probable. Il faudra passer l'argument 'axis = 2' pour que l'argmax soit calculée sur chaque élément des séquences. Stocker le résultat dans la variable **y_pred**."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "z09W6dSCFs0h"
      },
      "source": [
        "## Insérez votre code ici\n",
        "y_prob = model.predict (X_test[:100] )\n",
        "y_prob = y_prob.argmax (axis=2)\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZtXirCzdFs0h"
      },
      "source": [
        "#@title Solution\n",
        "y_prob = model.predict(X_test[:100])\n",
        "y_pred = y_prob.argmax(axis=2)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lYxbM1BmFs0h"
      },
      "source": [
        "* Exécuter la cellule suivante pour afficher le résulat du modèle sur les 100 premières phrase de X_test."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UH5a9Z1KFs0h"
      },
      "source": [
        "def target2text(X_int, y_pred):\n",
        "    X = []\n",
        "    for i in range(len(y_pred)):\n",
        "        text=''\n",
        "        for j in np.flip(np.arange(len(y_pred[0]))):\n",
        "            if y_pred[i, j]==1 :\n",
        "                text = ' ' + text\n",
        "            text = idx2char[X_int[i, j]] + text\n",
        "        X.append(text)\n",
        "    return X\n",
        "\n",
        "print('Prediction :')\n",
        "target2text(X_test[:len(y_pred)], y_pred)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4eW1ZaA7Fs0h"
      },
      "source": [
        "* Exécuter la cellule suivante pour afficher le résultat du modèle sur la phrase <i>\"thedatascienceismyway.\"</i>."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HgwBrIFDFs0i"
      },
      "source": [
        "def process_data(sentences, max_len=None):\n",
        "    if max_len==None:\n",
        "        max_len = max_length(sentences)\n",
        "    text_as_int = np.zeros([len(sentences), max_len], dtype=int)\n",
        "    for i, s in enumerate(sentences):\n",
        "        for j, c in enumerate(s):\n",
        "            text_as_int[i,j] = char2idx[c]\n",
        "    return text_as_int, max_len\n",
        "\n",
        "sentences_inputs, _ = process_data(['thedatascienceismyway!'], max_length_inputs)\n",
        "y_pred = model.predict(sentences_inputs)\n",
        "target2text(sentences_inputs, y_pred.argmax(axis=2))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WH40MtyFFs0i"
      },
      "source": [
        "## Conclusion\n",
        "\n",
        "> Cet exercice a essentiellement pour but de s'exercer à l'implémentation un modèle récurant. En effet, trouver les espaces entre les mots n'a pas d'application réelle.\n",
        ">\n",
        "> Toutefois, nous pouvons entrevoir les possibilités et la puissance du deep learning sur des tâches liées au texte."
      ]
    }
  ]
}