利用RNN_LSTM进行情感分析.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.3"
    },
    "colab": {
      "name": "利用RNN-LSTM进行情感分析.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/shiyadong123/hello-world/blob/master/%E5%88%A9%E7%94%A8RNN_LSTM%E8%BF%9B%E8%A1%8C%E6%83%85%E6%84%9F%E5%88%86%E6%9E%90.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xteoqHK_nXIB",
        "colab_type": "text"
      },
      "source": [
        "# 利用RNN-LSTM进行情感分析"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xnDSoS-RnXID",
        "colab_type": "text"
      },
      "source": [
        "# 1. RNN简介"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_hrN27HAnXIE",
        "colab_type": "text"
      },
      "source": [
        "## 1.1 RNN模型"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2f0npg9EnXIG",
        "colab_type": "text"
      },
      "source": [
        "循环神经网络（Recurrent Neural Network，RNN）是用来建模序列化数据的一 种主流深度学习模型。我们知道，传统的前馈神经网络一般的输入都是一个定长 的向量，无法处理变长的序列信息，即使通过一些方法把序列处理成定长的向量，模型也很难捕捉序列中的长距离依赖关系。RNN则通过将神经元串行起来处理序列化的数据。由于每个神经元能用它的内部变量保存之前输入的序列信息， 因此整个序列被浓缩成抽象的表示，并可以据此进行分类或生成新的序列。近年来，得益于计算能力的大幅提升和模型的改进，RNN在很多领域取得了突破性的进展——机器翻译、序列标注、图像描述、推荐系统、智能聊天机器人、自动作词作曲等。<br>\n",
        "\n",
        "下图即为一个典型的RNN网络结构：<br>\n",
        "一个长度为$T$的序列用RNN建模，展开之后可以看作是一个$T$层的前馈神经网络。其中，第$t$层的隐含状态$h_t$编码了序列中前$t$个输入的信息，可以通过当前的输入$x_t$和上一层神经网络的状态$h_{T-1}$计算得到；最后一层的状态$h_{T}$编码了整个序列的信息。\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EUdcOkHOnXIH",
        "colab_type": "text"
      },
      "source": [
        "![png](https://github.com/shiyadong123/Myimage/blob/master/RNN00.PNG?raw=true)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ptfuCaaOnXII",
        "colab_type": "text"
      },
      "source": [
        "## 1.2 LSTM简介"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qHRjP9r9nXIK",
        "colab_type": "text"
      },
      "source": [
        "+ RNN模型的缺陷\n",
        "\n",
        "RNN模型的求解采用BPTT（Back Propagation Through Time，基于时间的反向传播）算法实现，BPTT实际上是反向传播算法的简单变种。如果将RNN按时间展开成T层的前馈神经网络来理解，就和普通的反向传播算法非常类似。然而实践发现，使用BPTT算法学习的RNN模型并不能成功捕捉到长距离的依赖关系，这一现象主要源于深度神经网络中的**梯度消失**。<br>\n",
        "\n",
        "对于普通的前馈网络来说，梯度消失意味着无法通过加深网络层次来改善神经网络的预测效果，因为无论如何加深网络，*只有靠近输出的若干层才真正起到学习的作用*，这使得RNN模型很难学习到输入序列中的长距离依赖关系。<br>\n",
        "\n",
        "而LSTM模型通过加入**门控机制**，很大程度上弥补了梯度消失所带来的损失。\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AAXoL8YZnXIL",
        "colab_type": "text"
      },
      "source": [
        "![jpq](https://github.com/shiyadong123/Myimage/blob/master/lstm03.jpg?raw=true)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rs_hokAonXIM",
        "colab_type": "text"
      },
      "source": [
        "+ LSTM模型与一般RNN模型的比较\n",
        "\n",
        "如上图所示，左图是一般的RNN模型，右图是LSTM模型。<br>\n",
        "普通RNN模型，只是接收上一步的输出和当前步的输入，通过tanh激活函数，获得当前输出并传递到下一步。<br>\n",
        "而在LSTM模型中，每个$\\sigma$都是一个乘法门，决定输入的特征是否重要。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "N6NtsXpenXIO",
        "colab_type": "text"
      },
      "source": [
        "+ LSTM模型的结构\n",
        "\n",
        "如下图所示，红框从左至右分别是**遗忘门**、**输入门**、**输出门**<br>\n",
        "遗忘门：决定从cell状态中丢弃（或遗忘）什么信息，通过当前时刻的输入与前一个时刻的输出决定<br>\n",
        "输入门：确定并更新新信息到当前时刻的cell状态中<br>\n",
        "输出门：基于目前的cell状态决定该时刻的输出"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "seNSSbGenXIP",
        "colab_type": "text"
      },
      "source": [
        "![png](https://github.com/shiyadong123/Myimage/blob/master/lstm04.png?raw=true)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AvR_4RzUnXIR",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import pandas as pd\n",
        "import numpy as np"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LlhglmLdnXIW",
        "colab_type": "code",
        "outputId": "8569f207-ec05-47f9-f1aa-c86053dfa3ee",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "source": [
        "data = pd.read_csv('./Tweets.csv',usecols = ['text','airline_sentiment'])\n",
        "data.head()"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>airline_sentiment</th>\n",
              "      <th>text</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>neutral</td>\n",
              "      <td>@VirginAmerica What @dhepburn said.</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>positive</td>\n",
              "      <td>@VirginAmerica plus you've added commercials t...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>neutral</td>\n",
              "      <td>@VirginAmerica I didn't today... Must mean I n...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>negative</td>\n",
              "      <td>@VirginAmerica it's really aggressive to blast...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>negative</td>\n",
              "      <td>@VirginAmerica and it's a really big bad thing...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "  airline_sentiment                                               text\n",
              "0           neutral                @VirginAmerica What @dhepburn said.\n",
              "1          positive  @VirginAmerica plus you've added commercials t...\n",
              "2           neutral  @VirginAmerica I didn't today... Must mean I n...\n",
              "3          negative  @VirginAmerica it's really aggressive to blast...\n",
              "4          negative  @VirginAmerica and it's a really big bad thing..."
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 2
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zd_X3gGInXId",
        "colab_type": "code",
        "outputId": "da75092f-a2b1-4c01-e381-0cf831a2aff2",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "source": [
        "# 使用“0”“1”“2”对应“neutral”“positive”“negative”\n",
        "data.airline_sentiment = data.airline_sentiment.map({'neutral':0, 'positive':1,'negative':2})\n",
        "data.head()"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>airline_sentiment</th>\n",
              "      <th>text</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>@VirginAmerica What @dhepburn said.</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>@VirginAmerica plus you've added commercials t...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0</td>\n",
              "      <td>@VirginAmerica I didn't today... Must mean I n...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>2</td>\n",
              "      <td>@VirginAmerica it's really aggressive to blast...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>2</td>\n",
              "      <td>@VirginAmerica and it's a really big bad thing...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   airline_sentiment                                               text\n",
              "0                  0                @VirginAmerica What @dhepburn said.\n",
              "1                  1  @VirginAmerica plus you've added commercials t...\n",
              "2                  0  @VirginAmerica I didn't today... Must mean I n...\n",
              "3                  2  @VirginAmerica it's really aggressive to blast...\n",
              "4                  2  @VirginAmerica and it's a really big bad thing..."
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8p7U2cNanXIi",
        "colab_type": "code",
        "outputId": "84b24764-6c32-41d2-8dd9-f2a41787242f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "source": [
        "# 对评论做基本清洗，包括转换成小写、去掉特殊符号、分词\n",
        "import re\n",
        "data['text'] = data['text'].apply(lambda x: x.lower())\n",
        "data['text'] = data['text'].apply((lambda x: re.sub('[^a-zA-z0-9\\s]','',x)))\n",
        "data['text'] = data['text'].apply(lambda x: x.split())\n",
        "data.head()"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>airline_sentiment</th>\n",
              "      <th>text</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>[virginamerica, what, dhepburn, said]</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>[virginamerica, plus, youve, added, commercial...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0</td>\n",
              "      <td>[virginamerica, i, didnt, today, must, mean, i...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>2</td>\n",
              "      <td>[virginamerica, its, really, aggressive, to, b...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>2</td>\n",
              "      <td>[virginamerica, and, its, a, really, big, bad,...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   airline_sentiment                                               text\n",
              "0                  0              [virginamerica, what, dhepburn, said]\n",
              "1                  1  [virginamerica, plus, youve, added, commercial...\n",
              "2                  0  [virginamerica, i, didnt, today, must, mean, i...\n",
              "3                  2  [virginamerica, its, really, aggressive, to, b...\n",
              "4                  2  [virginamerica, and, its, a, really, big, bad,..."
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nmJmVQ-anXIm",
        "colab_type": "code",
        "outputId": "cb431a2b-b518-462b-9608-136ac6f8e58f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "source": [
        "data['length'] = data['text'].apply(len)\n",
        "data.head()"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>airline_sentiment</th>\n",
              "      <th>text</th>\n",
              "      <th>length</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>[virginamerica, what, dhepburn, said]</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>[virginamerica, plus, youve, added, commercial...</td>\n",
              "      <td>9</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0</td>\n",
              "      <td>[virginamerica, i, didnt, today, must, mean, i...</td>\n",
              "      <td>12</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>2</td>\n",
              "      <td>[virginamerica, its, really, aggressive, to, b...</td>\n",
              "      <td>17</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>2</td>\n",
              "      <td>[virginamerica, and, its, a, really, big, bad,...</td>\n",
              "      <td>10</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   airline_sentiment                                               text  length\n",
              "0                  0              [virginamerica, what, dhepburn, said]       4\n",
              "1                  1  [virginamerica, plus, youve, added, commercial...       9\n",
              "2                  0  [virginamerica, i, didnt, today, must, mean, i...      12\n",
              "3                  2  [virginamerica, its, really, aggressive, to, b...      17\n",
              "4                  2  [virginamerica, and, its, a, really, big, bad,...      10"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1OPjiG45nXIq",
        "colab_type": "code",
        "outputId": "623dd431-a590-4ef3-9b00-484a02ce330f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 179
        }
      },
      "source": [
        "data.length.describe()"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "count    14640.000000\n",
              "mean        17.500068\n",
              "std          6.871166\n",
              "min          2.000000\n",
              "25%         12.000000\n",
              "50%         19.000000\n",
              "75%         23.000000\n",
              "max         35.000000\n",
              "Name: length, dtype: float64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "q1OQr7UwnXIw",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# 将data中的评论全部合并到同一个list中，便于下面操作\n",
        "text = data['text'].sum()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "N9bvKPvtnXI1",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from collections import Counter\n",
        "# 统计text中词出现的频率 \n",
        "counts = Counter(text)\n",
        "# 按照词出现的频率进行降序排列，并生成一个单词到索引的字典\n",
        "vocab = sorted(counts, key=counts.get, reverse=True)\n",
        "vocab_to_int = {word: ii for ii, word in enumerate(vocab, 1)}"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": false,
        "id": "hLy2OvP0nXI6",
        "colab_type": "code",
        "outputId": "8122acb1-d511-4c3f-f77b-3c4e1ee07928",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "source": [
        "vocab_to_int"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'to': 1,\n",
              " 'the': 2,\n",
              " 'i': 3,\n",
              " 'a': 4,\n",
              " 'united': 5,\n",
              " 'you': 6,\n",
              " 'for': 7,\n",
              " 'flight': 8,\n",
              " 'on': 9,\n",
              " 'and': 10,\n",
              " 'my': 11,\n",
              " 'usairways': 12,\n",
              " 'americanair': 13,\n",
              " 'is': 14,\n",
              " 'in': 15,\n",
              " 'southwestair': 16,\n",
              " 'jetblue': 17,\n",
              " 'of': 18,\n",
              " 'me': 19,\n",
              " 'it': 20,\n",
              " 'your': 21,\n",
              " 'have': 22,\n",
              " 'was': 23,\n",
              " 'not': 24,\n",
              " 'with': 25,\n",
              " 'that': 26,\n",
              " 'at': 27,\n",
              " 'no': 28,\n",
              " 'this': 29,\n",
              " 'get': 30,\n",
              " 'but': 31,\n",
              " 'be': 32,\n",
              " 'from': 33,\n",
              " 'can': 34,\n",
              " 'are': 35,\n",
              " 'thanks': 36,\n",
              " 'cancelled': 37,\n",
              " 'we': 38,\n",
              " 'now': 39,\n",
              " 'an': 40,\n",
              " 'just': 41,\n",
              " 'service': 42,\n",
              " 'do': 43,\n",
              " 'so': 44,\n",
              " 'been': 45,\n",
              " 'help': 46,\n",
              " 'time': 47,\n",
              " 'im': 48,\n",
              " 'will': 49,\n",
              " 'customer': 50,\n",
              " 'up': 51,\n",
              " 'out': 52,\n",
              " 'our': 53,\n",
              " 'they': 54,\n",
              " 'us': 55,\n",
              " 'hours': 56,\n",
              " 'what': 57,\n",
              " 'when': 58,\n",
              " 'flights': 59,\n",
              " '2': 60,\n",
              " 'amp': 61,\n",
              " 'hold': 62,\n",
              " 'how': 63,\n",
              " 'its': 64,\n",
              " 'if': 65,\n",
              " 'plane': 66,\n",
              " 'all': 67,\n",
              " 'thank': 68,\n",
              " 'why': 69,\n",
              " 'cant': 70,\n",
              " 'there': 71,\n",
              " 'still': 72,\n",
              " 'please': 73,\n",
              " 'one': 74,\n",
              " 'need': 75,\n",
              " 'would': 76,\n",
              " 'delayed': 77,\n",
              " 'virginamerica': 78,\n",
              " 'back': 79,\n",
              " 'gate': 80,\n",
              " 'about': 81,\n",
              " 'had': 82,\n",
              " 'flightled': 83,\n",
              " 'call': 84,\n",
              " 'dont': 85,\n",
              " 'or': 86,\n",
              " 'as': 87,\n",
              " 'bag': 88,\n",
              " 'has': 89,\n",
              " 'after': 90,\n",
              " 'hour': 91,\n",
              " 'got': 92,\n",
              " 'am': 93,\n",
              " 'any': 94,\n",
              " 'like': 95,\n",
              " 'late': 96,\n",
              " 'phone': 97,\n",
              " 'today': 98,\n",
              " 'more': 99,\n",
              " 'over': 100,\n",
              " 'airline': 101,\n",
              " 'by': 102,\n",
              " 'again': 103,\n",
              " 'guys': 104,\n",
              " 'were': 105,\n",
              " 'fly': 106,\n",
              " 'waiting': 107,\n",
              " 'know': 108,\n",
              " 'trying': 109,\n",
              " 'airport': 110,\n",
              " 'way': 111,\n",
              " 'only': 112,\n",
              " 'ive': 113,\n",
              " 'great': 114,\n",
              " 'did': 115,\n",
              " 'then': 116,\n",
              " 'u': 117,\n",
              " 'going': 118,\n",
              " 'day': 119,\n",
              " 'should': 120,\n",
              " 'because': 121,\n",
              " 'wait': 122,\n",
              " 'never': 123,\n",
              " '3': 124,\n",
              " 'flying': 125,\n",
              " 'make': 126,\n",
              " 'change': 127,\n",
              " 'go': 128,\n",
              " 'last': 129,\n",
              " 'weather': 130,\n",
              " 'tomorrow': 131,\n",
              " 'really': 132,\n",
              " 'good': 133,\n",
              " 'delay': 134,\n",
              " 'home': 135,\n",
              " 'even': 136,\n",
              " 'off': 137,\n",
              " 'minutes': 138,\n",
              " '4': 139,\n",
              " 'people': 140,\n",
              " 'here': 141,\n",
              " 'bags': 142,\n",
              " 'another': 143,\n",
              " 'seat': 144,\n",
              " 'want': 145,\n",
              " 'new': 146,\n",
              " 'told': 147,\n",
              " 'take': 148,\n",
              " 'check': 149,\n",
              " 'first': 150,\n",
              " 'very': 151,\n",
              " 'see': 152,\n",
              " 'luggage': 153,\n",
              " 'agent': 154,\n",
              " 'them': 155,\n",
              " 'dm': 156,\n",
              " 'than': 157,\n",
              " 'ticket': 158,\n",
              " 'ever': 159,\n",
              " 'getting': 160,\n",
              " 'due': 161,\n",
              " 'could': 162,\n",
              " 'number': 163,\n",
              " 'love': 164,\n",
              " 'lost': 165,\n",
              " 'worst': 166,\n",
              " 'yes': 167,\n",
              " 'too': 168,\n",
              " 'email': 169,\n",
              " 'travel': 170,\n",
              " 'work': 171,\n",
              " 'thats': 172,\n",
              " 'next': 173,\n",
              " 'someone': 174,\n",
              " 'much': 175,\n",
              " 'hrs': 176,\n",
              " 'crew': 177,\n",
              " 'baggage': 178,\n",
              " 'flighted': 179,\n",
              " 'two': 180,\n",
              " 'days': 181,\n",
              " 'through': 182,\n",
              " 'who': 183,\n",
              " 'made': 184,\n",
              " 'didnt': 185,\n",
              " 'she': 186,\n",
              " 'well': 187,\n",
              " 'their': 188,\n",
              " 'right': 189,\n",
              " 'before': 190,\n",
              " 'trip': 191,\n",
              " 'seats': 192,\n",
              " 'other': 193,\n",
              " 'aa': 194,\n",
              " 'response': 195,\n",
              " 'being': 196,\n",
              " 'experience': 197,\n",
              " 'problems': 198,\n",
              " 'let': 199,\n",
              " 'some': 200,\n",
              " 'online': 201,\n",
              " 'sitting': 202,\n",
              " 'her': 203,\n",
              " 'best': 204,\n",
              " 'sent': 205,\n",
              " 'passengers': 206,\n",
              " 'bad': 207,\n",
              " 'does': 208,\n",
              " 'staff': 209,\n",
              " 'customers': 210,\n",
              " 'wont': 211,\n",
              " 'boarding': 212,\n",
              " 'min': 213,\n",
              " 'better': 214,\n",
              " 'long': 215,\n",
              " 'already': 216,\n",
              " 'doesnt': 217,\n",
              " 'said': 218,\n",
              " '1': 219,\n",
              " 'ill': 220,\n",
              " 'sure': 221,\n",
              " 'left': 222,\n",
              " 'jfk': 223,\n",
              " 'line': 224,\n",
              " 'where': 225,\n",
              " 'since': 226,\n",
              " 'youre': 227,\n",
              " 'book': 228,\n",
              " 'times': 229,\n",
              " 'booked': 230,\n",
              " 'stuck': 231,\n",
              " '5': 232,\n",
              " 'give': 233,\n",
              " 'morning': 234,\n",
              " 'into': 235,\n",
              " 'miss': 236,\n",
              " 'reservation': 237,\n",
              " 'same': 238,\n",
              " 'flightr': 239,\n",
              " 'tonight': 240,\n",
              " 'airlines': 241,\n",
              " 'whats': 242,\n",
              " 'miles': 243,\n",
              " 'think': 244,\n",
              " 'website': 245,\n",
              " 'find': 246,\n",
              " 'agents': 247,\n",
              " 'yet': 248,\n",
              " 'fleek': 249,\n",
              " 'nothing': 250,\n",
              " 'use': 251,\n",
              " 'issue': 252,\n",
              " 'w': 253,\n",
              " 'says': 254,\n",
              " 'care': 255,\n",
              " 'booking': 256,\n",
              " 'connection': 257,\n",
              " 'night': 258,\n",
              " 'refund': 259,\n",
              " 'dfw': 260,\n",
              " 'system': 261,\n",
              " 'fleets': 262,\n",
              " 'info': 263,\n",
              " 'flt': 264,\n",
              " 'he': 265,\n",
              " 'tell': 266,\n",
              " 'tried': 267,\n",
              " 'hotel': 268,\n",
              " 'delays': 269,\n",
              " 'put': 270,\n",
              " 'rebooked': 271,\n",
              " 'air': 272,\n",
              " 'hope': 273,\n",
              " 'keep': 274,\n",
              " 'pay': 275,\n",
              " 'called': 276,\n",
              " 'anything': 277,\n",
              " 'rude': 278,\n",
              " 'also': 279,\n",
              " 'missed': 280,\n",
              " 'nice': 281,\n",
              " 'down': 282,\n",
              " 'free': 283,\n",
              " 'done': 284,\n",
              " 'ago': 285,\n",
              " 'tickets': 286,\n",
              " 'follow': 287,\n",
              " 'rebook': 288,\n",
              " 'finally': 289,\n",
              " 'wifi': 290,\n",
              " '30': 291,\n",
              " 'class': 292,\n",
              " 'mins': 293,\n",
              " 'awesome': 294,\n",
              " 'issues': 295,\n",
              " 'able': 296,\n",
              " 'lax': 297,\n",
              " 'credit': 298,\n",
              " 'checked': 299,\n",
              " 'business': 300,\n",
              " 'every': 301,\n",
              " 'week': 302,\n",
              " 'working': 303,\n",
              " 'until': 304,\n",
              " 'appreciate': 305,\n",
              " 'having': 306,\n",
              " 'anyone': 307,\n",
              " 'come': 308,\n",
              " 'say': 309,\n",
              " 'missing': 310,\n",
              " 'looking': 311,\n",
              " '6': 312,\n",
              " 'phl': 313,\n",
              " 'board': 314,\n",
              " 'delta': 315,\n",
              " 'available': 316,\n",
              " 'via': 317,\n",
              " 'id': 318,\n",
              " '10': 319,\n",
              " 'app': 320,\n",
              " 'update': 321,\n",
              " 'always': 322,\n",
              " 'person': 323,\n",
              " 'status': 324,\n",
              " 'without': 325,\n",
              " 'problem': 326,\n",
              " 'answer': 327,\n",
              " 'couldnt': 328,\n",
              " 'making': 329,\n",
              " 'thx': 330,\n",
              " 'leave': 331,\n",
              " 'many': 332,\n",
              " 'team': 333,\n",
              " 'which': 334,\n",
              " 'isnt': 335,\n",
              " 'terrible': 336,\n",
              " 'yesterday': 337,\n",
              " 'understand': 338,\n",
              " 'voucher': 339,\n",
              " 'havent': 340,\n",
              " 'upgrade': 341,\n",
              " 'sorry': 342,\n",
              " 'helpful': 343,\n",
              " 'look': 344,\n",
              " 'ok': 345,\n",
              " 'fail': 346,\n",
              " 'dca': 347,\n",
              " 'planes': 348,\n",
              " 'amazing': 349,\n",
              " '1st': 350,\n",
              " 'sfo': 351,\n",
              " 'disappointed': 352,\n",
              " 'extra': 353,\n",
              " 'instead': 354,\n",
              " 'speak': 355,\n",
              " 'bc': 356,\n",
              " 'early': 357,\n",
              " 'name': 358,\n",
              " 'fix': 359,\n",
              " 'send': 360,\n",
              " '15': 361,\n",
              " '20': 362,\n",
              " 'rep': 363,\n",
              " 'southwest': 364,\n",
              " 'wasnt': 365,\n",
              " 'almost': 366,\n",
              " 'year': 367,\n",
              " 'money': 368,\n",
              " 'claim': 369,\n",
              " 'hi': 370,\n",
              " 'paid': 371,\n",
              " 'open': 372,\n",
              " 'pilot': 373,\n",
              " 'pass': 374,\n",
              " 'though': 375,\n",
              " 'earlier': 376,\n",
              " 'contact': 377,\n",
              " 'rt': 378,\n",
              " 'happy': 379,\n",
              " 'departure': 380,\n",
              " 'took': 381,\n",
              " 'ord': 382,\n",
              " 'different': 383,\n",
              " 'while': 384,\n",
              " 'try': 385,\n",
              " 'talk': 386,\n",
              " 'clt': 387,\n",
              " 'supposed': 388,\n",
              " 'connecting': 389,\n",
              " 'site': 390,\n",
              " 'vegas': 391,\n",
              " 'hung': 392,\n",
              " 'unacceptable': 393,\n",
              " 'received': 394,\n",
              " 'attendant': 395,\n",
              " 'stop': 396,\n",
              " 'least': 397,\n",
              " 'something': 398,\n",
              " '12': 399,\n",
              " 'employees': 400,\n",
              " 'airways': 401,\n",
              " 'job': 402,\n",
              " 'full': 403,\n",
              " 'actually': 404,\n",
              " 'yall': 405,\n",
              " 'direct': 406,\n",
              " 'boston': 407,\n",
              " 'poor': 408,\n",
              " 'ridiculous': 409,\n",
              " 'stranded': 410,\n",
              " 'ground': 411,\n",
              " 'dallas': 412,\n",
              " 'card': 413,\n",
              " 'horrible': 414,\n",
              " 'company': 415,\n",
              " 'tarmac': 416,\n",
              " 'add': 417,\n",
              " 'show': 418,\n",
              " 'oh': 419,\n",
              " 'his': 420,\n",
              " 'destinationdragons': 421,\n",
              " 'family': 422,\n",
              " 'everyone': 423,\n",
              " 'chicago': 424,\n",
              " 'those': 425,\n",
              " 'denver': 426,\n",
              " '25': 427,\n",
              " '45': 428,\n",
              " 'seriously': 429,\n",
              " 'landed': 430,\n",
              " 'calling': 431,\n",
              " 'twitter': 432,\n",
              " 'old': 433,\n",
              " 'taking': 434,\n",
              " 'found': 435,\n",
              " 'weeks': 436,\n",
              " 'ur': 437,\n",
              " 'ewr': 438,\n",
              " 'start': 439,\n",
              " 'leaving': 440,\n",
              " 'needs': 441,\n",
              " 'doing': 442,\n",
              " 'snow': 443,\n",
              " 'chance': 444,\n",
              " 'american': 445,\n",
              " 'most': 446,\n",
              " 'san': 447,\n",
              " 'maybe': 448,\n",
              " '8': 449,\n",
              " 'wrong': 450,\n",
              " 'tweet': 451,\n",
              " '7': 452,\n",
              " 'wife': 453,\n",
              " 'far': 454,\n",
              " 'reason': 455,\n",
              " 'might': 456,\n",
              " 'policy': 457,\n",
              " 'bos': 458,\n",
              " 'gave': 459,\n",
              " 'food': 460,\n",
              " 'account': 461,\n",
              " 'point': 462,\n",
              " 'vacation': 463,\n",
              " 'away': 464,\n",
              " 'reply': 465,\n",
              " 'half': 466,\n",
              " 'enough': 467,\n",
              " 'message': 468,\n",
              " 'big': 469,\n",
              " 'sit': 470,\n",
              " 'both': 471,\n",
              " 'coming': 472,\n",
              " 'return': 473,\n",
              " 'hey': 474,\n",
              " 'fee': 475,\n",
              " 'once': 476,\n",
              " 'desk': 477,\n",
              " 'past': 478,\n",
              " 'charge': 479,\n",
              " 'theres': 480,\n",
              " 'reservations': 481,\n",
              " 'three': 482,\n",
              " 'option': 483,\n",
              " 'frustrated': 484,\n",
              " 'mechanical': 485,\n",
              " 'hr': 486,\n",
              " 'little': 487,\n",
              " 'thing': 488,\n",
              " 'may': 489,\n",
              " 'together': 490,\n",
              " 'went': 491,\n",
              " 'confirmation': 492,\n",
              " 'car': 493,\n",
              " 'these': 494,\n",
              " '40': 495,\n",
              " 'using': 496,\n",
              " 'changed': 497,\n",
              " 'offer': 498,\n",
              " 'asked': 499,\n",
              " 'twice': 500,\n",
              " 'nyc': 501,\n",
              " 'soon': 502,\n",
              " 'link': 503,\n",
              " 'question': 504,\n",
              " 'kids': 505,\n",
              " 'broken': 506,\n",
              " '24': 507,\n",
              " 'him': 508,\n",
              " 'charlotte': 509,\n",
              " 'cool': 510,\n",
              " 'possible': 511,\n",
              " 'destination': 512,\n",
              " 'less': 513,\n",
              " 'stay': 514,\n",
              " 'saying': 515,\n",
              " 'weve': 516,\n",
              " 'around': 517,\n",
              " 'pls': 518,\n",
              " 'used': 519,\n",
              " 'things': 520,\n",
              " 'looks': 521,\n",
              " 'longer': 522,\n",
              " 'makes': 523,\n",
              " 'few': 524,\n",
              " 'runway': 525,\n",
              " 'lot': 526,\n",
              " 'traveling': 527,\n",
              " 'worse': 528,\n",
              " 'landing': 529,\n",
              " 'newark': 530,\n",
              " 'guess': 531,\n",
              " 'given': 532,\n",
              " 'ua': 533,\n",
              " 'awful': 534,\n",
              " 'terminal': 535,\n",
              " 'during': 536,\n",
              " 'cost': 537,\n",
              " '50': 538,\n",
              " 'real': 539,\n",
              " 'plus': 540,\n",
              " 'youve': 541,\n",
              " 'scheduled': 542,\n",
              " 'waited': 543,\n",
              " 'dc': 544,\n",
              " 'telling': 545,\n",
              " 'idea': 546,\n",
              " '200': 547,\n",
              " 'iad': 548,\n",
              " 'checkin': 549,\n",
              " 'houston': 550,\n",
              " 'attendants': 551,\n",
              " 'hear': 552,\n",
              " 'such': 553,\n",
              " 'heard': 554,\n",
              " 'seems': 555,\n",
              " 'row': 556,\n",
              " 'lack': 557,\n",
              " 'own': 558,\n",
              " 'else': 559,\n",
              " 'thru': 560,\n",
              " 'believe': 561,\n",
              " 'friend': 562,\n",
              " 'mean': 563,\n",
              " 'arrived': 564,\n",
              " 'happened': 565,\n",
              " 'hard': 566,\n",
              " 'life': 567,\n",
              " 'philly': 568,\n",
              " 'international': 569,\n",
              " 'frustrating': 570,\n",
              " 'sat': 571,\n",
              " 'member': 572,\n",
              " 'points': 573,\n",
              " 'sucks': 574,\n",
              " 'cust': 575,\n",
              " 'arrive': 576,\n",
              " 'wouldnt': 577,\n",
              " 'reflight': 578,\n",
              " 'phx': 579,\n",
              " 'feel': 580,\n",
              " 'options': 581,\n",
              " 'assistance': 582,\n",
              " 'request': 583,\n",
              " 'calls': 584,\n",
              " 'lol': 585,\n",
              " 'fll': 586,\n",
              " 'information': 587,\n",
              " 'maintenance': 588,\n",
              " 'monday': 589,\n",
              " 'hoping': 590,\n",
              " 'loyal': 591,\n",
              " 'pick': 592,\n",
              " 'second': 593,\n",
              " 'end': 594,\n",
              " 'everything': 595,\n",
              " 'lga': 596,\n",
              " 'minute': 597,\n",
              " 'quick': 598,\n",
              " 'glad': 599,\n",
              " 'cannot': 600,\n",
              " 'miami': 601,\n",
              " 'jet': 602,\n",
              " 'changes': 603,\n",
              " 'swa': 604,\n",
              " 'process': 605,\n",
              " 'forward': 606,\n",
              " 'expect': 607,\n",
              " 'joke': 608,\n",
              " 'arent': 609,\n",
              " '100': 610,\n",
              " 'error': 611,\n",
              " 'wanted': 612,\n",
              " 'standby': 613,\n",
              " 'each': 614,\n",
              " 'ask': 615,\n",
              " 'iah': 616,\n",
              " 'busy': 617,\n",
              " 'usair': 618,\n",
              " '2nd': 619,\n",
              " 'checking': 620,\n",
              " 'ceo': 621,\n",
              " 'room': 622,\n",
              " 'counting': 623,\n",
              " 'deal': 624,\n",
              " 'suck': 625,\n",
              " 'pilots': 626,\n",
              " 'thought': 627,\n",
              " 'reach': 628,\n",
              " 'apology': 629,\n",
              " 'theyre': 630,\n",
              " 'needed': 631,\n",
              " 'spent': 632,\n",
              " 'human': 633,\n",
              " 'bring': 634,\n",
              " 'paying': 635,\n",
              " 'lets': 636,\n",
              " 'years': 637,\n",
              " 'group': 638,\n",
              " 'boarded': 639,\n",
              " 'asking': 640,\n",
              " 'unitedairlines': 641,\n",
              " 'imaginedragons': 642,\n",
              " 'blue': 643,\n",
              " 'flew': 644,\n",
              " 'wish': 645,\n",
              " 'las': 646,\n",
              " 'world': 647,\n",
              " 'entire': 648,\n",
              " 'empty': 649,\n",
              " 'computer': 650,\n",
              " 'between': 651,\n",
              " 'form': 652,\n",
              " 'guy': 653,\n",
              " 'complaint': 654,\n",
              " 'case': 655,\n",
              " 'sw': 656,\n",
              " 'club': 657,\n",
              " 'la': 658,\n",
              " 'award': 659,\n",
              " 'yeah': 660,\n",
              " 'bought': 661,\n",
              " 'provide': 662,\n",
              " 'buy': 663,\n",
              " 'lose': 664,\n",
              " '800': 665,\n",
              " 'passenger': 666,\n",
              " 'place': 667,\n",
              " 'carry': 668,\n",
              " 'currently': 669,\n",
              " 'address': 670,\n",
              " 'fees': 671,\n",
              " 'months': 672,\n",
              " 'taken': 673,\n",
              " 'pretty': 674,\n",
              " 'city': 675,\n",
              " 'happens': 676,\n",
              " '9': 677,\n",
              " 'under': 678,\n",
              " 'updates': 679,\n",
              " 'helping': 680,\n",
              " 'whole': 681,\n",
              " 'flighting': 682,\n",
              " 'drive': 683,\n",
              " 'run': 684,\n",
              " 'communication': 685,\n",
              " 'leg': 686,\n",
              " 'counter': 687,\n",
              " 'bwi': 688,\n",
              " 'sunday': 689,\n",
              " 'hopefully': 690,\n",
              " 'either': 691,\n",
              " 'wow': 692,\n",
              " 'asap': 693,\n",
              " 'tsa': 694,\n",
              " 'respond': 695,\n",
              " 'rather': 696,\n",
              " 'seem': 697,\n",
              " 'happen': 698,\n",
              " 'den': 699,\n",
              " 'live': 700,\n",
              " 'beyond': 701,\n",
              " 'moved': 702,\n",
              " 'tv': 703,\n",
              " 'wtf': 704,\n",
              " 'price': 705,\n",
              " 'future': 706,\n",
              " 'supervisor': 707,\n",
              " 'sleep': 708,\n",
              " 'fault': 709,\n",
              " 'flightlations': 710,\n",
              " 'appreciated': 711,\n",
              " 'companion': 712,\n",
              " 'holding': 713,\n",
              " 'friends': 714,\n",
              " 'support': 715,\n",
              " 'fine': 716,\n",
              " 'month': 717,\n",
              " 'super': 718,\n",
              " 'confirmed': 719,\n",
              " 'ready': 720,\n",
              " 'nashville': 721,\n",
              " 'automated': 722,\n",
              " 'correct': 723,\n",
              " 'inflight': 724,\n",
              " 'high': 725,\n",
              " 'shouldnt': 726,\n",
              " 'figure': 727,\n",
              " 'pm': 728,\n",
              " 'mco': 729,\n",
              " 'sad': 730,\n",
              " 'offered': 731,\n",
              " 'plans': 732,\n",
              " 'apparently': 733,\n",
              " 'access': 734,\n",
              " 'easy': 735,\n",
              " 'cabin': 736,\n",
              " 'safety': 737,\n",
              " 'upset': 738,\n",
              " 'okay': 739,\n",
              " 'situation': 740,\n",
              " 'date': 741,\n",
              " 'tuesday': 742,\n",
              " 'cause': 743,\n",
              " 'atlanta': 744,\n",
              " 'media': 745,\n",
              " 'customerservice': 746,\n",
              " 'flown': 747,\n",
              " 'completely': 748,\n",
              " 'handle': 749,\n",
              " 'treat': 750,\n",
              " 'probably': 751,\n",
              " 'set': 752,\n",
              " 'record': 753,\n",
              " 'atl': 754,\n",
              " 'hate': 755,\n",
              " 'following': 756,\n",
              " 'running': 757,\n",
              " 'reschedule': 758,\n",
              " 'luv': 759,\n",
              " 'r': 760,\n",
              " 'charged': 761,\n",
              " 'luck': 762,\n",
              " 'storm': 763,\n",
              " 'connections': 764,\n",
              " 'orlando': 765,\n",
              " 'top': 766,\n",
              " 'compensation': 767,\n",
              " 'original': 768,\n",
              " 'means': 769,\n",
              " 'huge': 770,\n",
              " 'gets': 771,\n",
              " 'switch': 772,\n",
              " 'disconnected': 773,\n",
              " 'kind': 774,\n",
              " 'must': 775,\n",
              " 'cold': 776,\n",
              " 'middle': 777,\n",
              " 'austin': 778,\n",
              " 'fare': 779,\n",
              " 'platinum': 780,\n",
              " '11': 781,\n",
              " 'details': 782,\n",
              " 'read': 783,\n",
              " 'zero': 784,\n",
              " 'members': 785,\n",
              " 'delivered': 786,\n",
              " 'employee': 787,\n",
              " 'fact': 788,\n",
              " 'seen': 789,\n",
              " 'allowed': 790,\n",
              " 'sending': 791,\n",
              " 'losing': 792,\n",
              " 'allow': 793,\n",
              " 'share': 794,\n",
              " 'shes': 795,\n",
              " 'giving': 796,\n",
              " 'priority': 797,\n",
              " 'gonna': 798,\n",
              " 'fun': 799,\n",
              " 'gives': 800,\n",
              " 'helped': 801,\n",
              " 'several': 802,\n",
              " 'purchase': 803,\n",
              " 'clothes': 804,\n",
              " 'changing': 805,\n",
              " 'schedule': 806,\n",
              " '2015': 807,\n",
              " 'gt': 808,\n",
              " 'flightlation': 809,\n",
              " 'came': 810,\n",
              " 'despite': 811,\n",
              " '1k': 812,\n",
              " 'shit': 813,\n",
              " 'part': 814,\n",
              " 'winter': 815,\n",
              " 'weekend': 816,\n",
              " 'extremely': 817,\n",
              " 'front': 818,\n",
              " 'myself': 819,\n",
              " 'crazy': 820,\n",
              " 'wall': 821,\n",
              " 'answering': 822,\n",
              " 'rock': 823,\n",
              " 'layover': 824,\n",
              " 'takes': 825,\n",
              " 'space': 826,\n",
              " 'spoke': 827,\n",
              " 'reps': 828,\n",
              " 'explain': 829,\n",
              " 'ppl': 830,\n",
              " 'aircraft': 831,\n",
              " 'lounge': 832,\n",
              " 'unable': 833,\n",
              " 'multiple': 834,\n",
              " 'sense': 835,\n",
              " 'water': 836,\n",
              " 'country': 837,\n",
              " 'birthday': 838,\n",
              " 'four': 839,\n",
              " 'mobile': 840,\n",
              " 'friday': 841,\n",
              " '35': 842,\n",
              " 'gold': 843,\n",
              " 'rescheduled': 844,\n",
              " 'land': 845,\n",
              " 'inconvenience': 846,\n",
              " 'tix': 847,\n",
              " 'complete': 848,\n",
              " 'word': 849,\n",
              " 'services': 850,\n",
              " 'knew': 851,\n",
              " 'goes': 852,\n",
              " 'held': 853,\n",
              " 'works': 854,\n",
              " 'arrival': 855,\n",
              " 'unfortunately': 856,\n",
              " 'fixed': 857,\n",
              " 'list': 858,\n",
              " 'rdu': 859,\n",
              " 'hello': 860,\n",
              " 'non': 861,\n",
              " 'lots': 862,\n",
              " 'totally': 863,\n",
              " 'afternoon': 864,\n",
              " 'report': 865,\n",
              " 'husband': 866,\n",
              " 'absolutely': 867,\n",
              " 'mileage': 868,\n",
              " 'overnight': 869,\n",
              " 'kudos': 870,\n",
              " 'load': 871,\n",
              " 'confirm': 872,\n",
              " 'nope': 873,\n",
              " 'answers': 874,\n",
              " 'phones': 875,\n",
              " 're': 876,\n",
              " 'treated': 877,\n",
              " 'hang': 878,\n",
              " 'hasnt': 879,\n",
              " 'true': 880,\n",
              " 'relations': 881,\n",
              " 'jetblues': 882,\n",
              " 'virgin': 883,\n",
              " 'news': 884,\n",
              " 'feb': 885,\n",
              " 'code': 886,\n",
              " 'friendly': 887,\n",
              " 'folks': 888,\n",
              " 'keeps': 889,\n",
              " 'route': 890,\n",
              " 'area': 891,\n",
              " 'round': 892,\n",
              " '3rd': 893,\n",
              " 'learn': 894,\n",
              " 'wonderful': 895,\n",
              " 'mine': 896,\n",
              " 'drop': 897,\n",
              " 'worth': 898,\n",
              " 'responding': 899,\n",
              " 'literally': 900,\n",
              " 'control': 901,\n",
              " 'b': 902,\n",
              " 'showing': 903,\n",
              " 'attitude': 904,\n",
              " 'social': 905,\n",
              " 'plan': 906,\n",
              " 'haha': 907,\n",
              " 'dividend': 908,\n",
              " 'plz': 909,\n",
              " 'page': 910,\n",
              " 'anyway': 911,\n",
              " 'closed': 912,\n",
              " 'spend': 913,\n",
              " 'americanairlines': 914,\n",
              " 'ny': 915,\n",
              " 'bit': 916,\n",
              " 'small': 917,\n",
              " 'course': 918,\n",
              " 'excellent': 919,\n",
              " 'saw': 920,\n",
              " 'overhead': 921,\n",
              " 'connect': 922,\n",
              " 'baby': 923,\n",
              " 'matter': 924,\n",
              " 'dealing': 925,\n",
              " 'lines': 926,\n",
              " 'letter': 927,\n",
              " 'mia': 928,\n",
              " 'flyer': 929,\n",
              " 'ride': 930,\n",
              " 'watch': 931,\n",
              " 'nonstop': 932,\n",
              " 'depart': 933,\n",
              " 'choice': 934,\n",
              " 'passbook': 935,\n",
              " 'evening': 936,\n",
              " 'submitted': 937,\n",
              " 'short': 938,\n",
              " '90': 939,\n",
              " 'wo': 940,\n",
              " 'anymore': 941,\n",
              " 'vouchers': 942,\n",
              " 'anywhere': 943,\n",
              " '22': 944,\n",
              " 'arriving': 945,\n",
              " 'note': 946,\n",
              " 'feedback': 947,\n",
              " 'offering': 948,\n",
              " 'neveragain': 949,\n",
              " 'letting': 950,\n",
              " 'however': 951,\n",
              " 'bna': 952,\n",
              " 'fl': 953,\n",
              " 'worries': 954,\n",
              " 'gone': 955,\n",
              " 'excited': 956,\n",
              " 'seating': 957,\n",
              " 'sign': 958,\n",
              " 'mind': 959,\n",
              " 'shows': 960,\n",
              " 'trouble': 961,\n",
              " 'saturday': 962,\n",
              " 'kept': 963,\n",
              " 'hes': 964,\n",
              " 'march': 965,\n",
              " 'behind': 966,\n",
              " 'forced': 967,\n",
              " 'child': 968,\n",
              " 'yr': 969,\n",
              " 'man': 970,\n",
              " 'move': 971,\n",
              " 'phoenix': 972,\n",
              " 'ice': 973,\n",
              " 'catch': 974,\n",
              " 'complaints': 975,\n",
              " '1hr': 976,\n",
              " 'usairwaysfail': 977,\n",
              " 'door': 978,\n",
              " 'failed': 979,\n",
              " 'mess': 980,\n",
              " 'entertainment': 981,\n",
              " 'takeoff': 982,\n",
              " 'btw': 983,\n",
              " 'web': 984,\n",
              " 'window': 985,\n",
              " 'started': 986,\n",
              " 'others': 987,\n",
              " 'domestic': 988,\n",
              " 'worked': 989,\n",
              " 'gates': 990,\n",
              " 'conf': 991,\n",
              " 'resolved': 992,\n",
              " 'followed': 993,\n",
              " 'ruined': 994,\n",
              " 'knows': 995,\n",
              " 'cs': 996,\n",
              " 'cover': 997,\n",
              " 'explanation': 998,\n",
              " 'bumped': 999,\n",
              " 'hangs': 1000,\n",
              " ...}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Z66PuliknXJA",
        "colab_type": "code",
        "outputId": "07372f56-9c60-4e83-c2bf-dfb77d95529c",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 71
        }
      },
      "source": [
        "# 提取出每一条评论，将评论的每一个word转换成vocab_to_int字典中对应的int索引，即生成词向量\n",
        "reviews_ints = []\n",
        "for review in data['text']:\n",
        "    reviews_ints.append([vocab_to_int[word] for word in review])\n",
        "\n",
        "# 字典中word数量\n",
        "print('Unique words: ', len((vocab_to_int)))\n",
        "# 展示一条评论被转换成词向量的效果\n",
        "print('Tokenized review: \\n', reviews_ints[:1])"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Unique words:  16722\n",
            "Tokenized review: \n",
            " [[78, 57, 6612, 218]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": false,
        "id": "GttWeTc6nXJG",
        "colab_type": "code",
        "outputId": "65a28cd9-036f-439d-ec60-64e83baebe70",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 881
        }
      },
      "source": [
        "reviews_ints[:10]"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[[78, 57, 6612, 218],\n",
              " [78, 540, 541, 1122, 2453, 1, 2, 197, 6613],\n",
              " [78, 3, 185, 98, 775, 563, 3, 75, 1, 148, 143, 191],\n",
              " [78,\n",
              "  64,\n",
              "  132,\n",
              "  3663,\n",
              "  1,\n",
              "  4598,\n",
              "  4599,\n",
              "  981,\n",
              "  15,\n",
              "  21,\n",
              "  3091,\n",
              "  3664,\n",
              "  61,\n",
              "  54,\n",
              "  22,\n",
              "  487,\n",
              "  2714],\n",
              " [78, 10, 64, 4, 132, 469, 207, 488, 81, 20],\n",
              " [78,\n",
              "  429,\n",
              "  76,\n",
              "  275,\n",
              "  291,\n",
              "  4,\n",
              "  8,\n",
              "  7,\n",
              "  192,\n",
              "  26,\n",
              "  185,\n",
              "  22,\n",
              "  29,\n",
              "  2051,\n",
              "  64,\n",
              "  132,\n",
              "  2,\n",
              "  112,\n",
              "  207,\n",
              "  488,\n",
              "  81,\n",
              "  125,\n",
              "  1781],\n",
              " [78, 167, 1427, 301, 47, 3, 106, 2221, 29, 4600, 6614, 211, 128, 464],\n",
              " [78, 132, 280, 4, 3092, 1690, 7, 3665, 325, 3666, 6615, 71, 6616],\n",
              " [78, 187, 3, 6617, 39, 3, 43, 1592],\n",
              " [78, 20, 23, 349, 10, 564, 40, 91, 357, 227, 168, 133, 1, 19]]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XtbeuiVlnXJM",
        "colab_type": "text"
      },
      "source": [
        "+ 虽然使用整值索引替代了评论中的word，但是由于评论的word数量不一致，有短有长，所以设置`seq_len=30`，保证每个长度都是30，不够补零，多的取前30个，我们使用keras中的`preprocessing`模块可以很方便的完成这个任务"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5m_Y2feMnXJO",
        "colab_type": "code",
        "outputId": "f92d28ef-3705-4bdb-94f8-3dc2f0f97855",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "seq_len = 30\n",
        "from tensorflow.contrib.keras import preprocessing\n",
        "features = np.zeros((len(reviews_ints),seq_len),dtype=int)\n",
        "#将reviews_ints值逐行 赋值给features\n",
        "features = preprocessing.sequence.pad_sequences(reviews_ints, seq_len)\n",
        "features.shape"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(14640, 30)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": false,
        "id": "NOOrIusmnXJU",
        "colab_type": "code",
        "outputId": "ea8c5742-fd9f-491f-87f8-e0892e92e929",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 143
        }
      },
      "source": [
        "features"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[    0,     0,     0, ...,    57,  6612,   218],\n",
              "       [    0,     0,     0, ...,     2,   197,  6613],\n",
              "       [    0,     0,     0, ...,   148,   143,   191],\n",
              "       ...,\n",
              "       [    0,     0,     0, ...,   241,     1, 16722],\n",
              "       [    0,     0,     0, ...,   126,    11,  2620],\n",
              "       [   13,    38,    22, ...,     2,   173,     8]], dtype=int32)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2KN_meZmnXJa",
        "colab_type": "code",
        "outputId": "aa751043-7134-42f0-97fd-f2793142149e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 89
        }
      },
      "source": [
        "# lables\n",
        "Y = pd.get_dummies(data['airline_sentiment']).values\n",
        "encoded_labels = Y\n",
        "\n",
        "# data拆分比例\n",
        "split_frac = 0.8\n",
        "\n",
        "# 将data拆分成训练、验证、测试三部分，包括整值索引与标签\n",
        "split_idx = int(len(features)*split_frac)\n",
        "train_x, remaining_x = features[:split_idx], features[split_idx:]\n",
        "train_y, remaining_y = encoded_labels[:split_idx], encoded_labels[split_idx:]\n",
        "\n",
        "# 其中验证与测试比例相同\n",
        "test_idx = int(len(remaining_x)*0.5)\n",
        "val_x, test_x = remaining_x[:test_idx], remaining_x[test_idx:]\n",
        "val_y, test_y = remaining_y[:test_idx], remaining_y[test_idx:]\n",
        " \n",
        "# 输出训练、验证、测试集的大小\n",
        "print(\"\\t\\t\\tFeature Shapes:\")\n",
        "print(\"Train set: \\t\\t{}\".format(train_x.shape), \n",
        "      \"\\nValidation set: \\t{}\".format(val_x.shape),\n",
        "      \"\\nTest set: \\t\\t{}\".format(test_x.shape))"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\t\t\tFeature Shapes:\n",
            "Train set: \t\t(11712, 30) \n",
            "Validation set: \t(1464, 30) \n",
            "Test set: \t\t(1464, 30)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uF9S35WTnXJf",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import torch\n",
        "from torch.utils.data import TensorDataset, DataLoader\n",
        "\n",
        "# 将训练、验证、测试集转换成Tensor\n",
        "train_data = TensorDataset(torch.from_numpy(np.array(train_x)), torch.from_numpy(np.array(train_y)))\n",
        "valid_data = TensorDataset(torch.from_numpy(np.array(val_x)), torch.from_numpy(np.array(val_y)))\n",
        "test_data = TensorDataset(torch.from_numpy(np.array(test_x)), torch.from_numpy(np.array(test_y)))\n",
        " \n",
        "batch_size = 50\n",
        "\n",
        "# 训练、验证、测试集 Dataloader\n",
        "train_loader = DataLoader(train_data, shuffle=True, batch_size=batch_size)\n",
        "valid_loader = DataLoader(valid_data, shuffle=True, batch_size=batch_size)\n",
        "test_loader = DataLoader(test_data, shuffle=True, batch_size=batch_size)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IZwxReoAnXJk",
        "colab_type": "code",
        "outputId": "4b3571bf-aa21-4586-9f33-e17919a6c5a4",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "source": [
        "# 展示一批训练数据\n",
        "dataiter = iter(train_loader)\n",
        "sample_x, sample_y = dataiter.next()\n",
        " \n",
        "print('Sample input size: ', sample_x.size()) # batch_size=50, seq_len=30\n",
        "print('Sample input: \\n', sample_x)\n",
        "\n",
        "print('Sample label size: ', sample_y.size()) # batch_size=50\n",
        "print('Sample label: \\n', sample_y)"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Sample input size:  torch.Size([50, 30])\n",
            "Sample input: \n",
            " tensor([[    0,     0,     0,  ...,   422,  3586,   693],\n",
            "        [    0,     0,     0,  ...,    29,    14,  1543],\n",
            "        [    0,     0,     0,  ...,  1044,    29,   936],\n",
            "        ...,\n",
            "        [    0,     0,     0,  ...,   167, 14490,    14],\n",
            "        [    0,     0,     0,  ...,     9,   249, 11762],\n",
            "        [    0,     0,     0,  ...,  7728,   420,  3287]], dtype=torch.int32)\n",
            "Sample label size:  torch.Size([50, 3])\n",
            "Sample label: \n",
            " tensor([[0, 0, 1],\n",
            "        [0, 0, 1],\n",
            "        [1, 0, 0],\n",
            "        [0, 0, 1],\n",
            "        [0, 1, 0],\n",
            "        [0, 1, 0],\n",
            "        [0, 0, 1],\n",
            "        [0, 1, 0],\n",
            "        [0, 0, 1],\n",
            "        [0, 0, 1],\n",
            "        [0, 1, 0],\n",
            "        [0, 0, 1],\n",
            "        [0, 0, 1],\n",
            "        [0, 1, 0],\n",
            "        [1, 0, 0],\n",
            "        [0, 1, 0],\n",
            "        [0, 0, 1],\n",
            "        [0, 0, 1],\n",
            "        [0, 1, 0],\n",
            "        [0, 0, 1],\n",
            "        [0, 1, 0],\n",
            "        [0, 0, 1],\n",
            "        [0, 0, 1],\n",
            "        [1, 0, 0],\n",
            "        [0, 1, 0],\n",
            "        [0, 0, 1],\n",
            "        [1, 0, 0],\n",
            "        [0, 1, 0],\n",
            "        [0, 0, 1],\n",
            "        [1, 0, 0],\n",
            "        [0, 0, 1],\n",
            "        [1, 0, 0],\n",
            "        [0, 0, 1],\n",
            "        [1, 0, 0],\n",
            "        [0, 0, 1],\n",
            "        [1, 0, 0],\n",
            "        [0, 0, 1],\n",
            "        [0, 0, 1],\n",
            "        [0, 0, 1],\n",
            "        [0, 0, 1],\n",
            "        [0, 0, 1],\n",
            "        [0, 1, 0],\n",
            "        [1, 0, 0],\n",
            "        [0, 1, 0],\n",
            "        [0, 1, 0],\n",
            "        [1, 0, 0],\n",
            "        [0, 0, 1],\n",
            "        [0, 1, 0],\n",
            "        [1, 0, 0],\n",
            "        [0, 0, 1]], dtype=torch.uint8)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": false,
        "id": "480kMJLKnXJr",
        "colab_type": "code",
        "outputId": "d97775e8-8930-4443-84c9-ed719ecd7448",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "# 检查有没有GPU\n",
        "train_on_gpu=torch.cuda.is_available()\n",
        " \n",
        "if(train_on_gpu):\n",
        "    print('Training on GPU.')\n",
        "else:\n",
        "    print('No GPU available, training on CPU.')"
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Training on GPU.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MvJwX3mhnXJ2",
        "colab_type": "text"
      },
      "source": [
        "使用pytorch搭建RNN网络,来完成情感分析的任务"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5dioorycnXJ6",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import torch.nn as nn\n",
        " \n",
        "class SentimentRNN(nn.Module):\n",
        " \n",
        "    def __init__(self, vocab_size, output_size, embedding_dim, hidden_dim, n_layers, bidirectional=True, drop_prob=0.5):\n",
        "       \n",
        "        super(SentimentRNN, self).__init__()\n",
        " \n",
        "#         self.output_size = output_size\n",
        "        self.n_layers = n_layers\n",
        "        self.hidden_dim = hidden_dim\n",
        "        # 双向RNN\n",
        "        self.bidirectional = bidirectional\n",
        "        \n",
        "        # 进行Embedding操作\n",
        "        self.embedding = nn.Embedding(vocab_size, embedding_dim)\n",
        "        # LSTM层\n",
        "        self.lstm = nn.LSTM(embedding_dim, hidden_dim, n_layers, \n",
        "                            dropout=drop_prob, batch_first=True,\n",
        "                            bidirectional=bidirectional)\n",
        "        \n",
        "        # dropout层\n",
        "        self.dropout = nn.Dropout(0.3)\n",
        "        \n",
        "        # 是否使用双向RNN\n",
        "        if bidirectional:\n",
        "          self.fc = nn.Linear(hidden_dim*2, output_size)\n",
        "        else:\n",
        "          self.fc = nn.Linear(hidden_dim, output_size)\n",
        "         \n",
        "        # softmax层\n",
        "        self.soft = nn.Softmax()\n",
        "        \n",
        " \n",
        "    def forward(self, x, hidden):\n",
        "       \n",
        "        batch_size = x.size(0)\n",
        " \n",
        "        x = x.long()\n",
        "        # Embedding层的输出\n",
        "        embeds = self.embedding(x)\n",
        "        # LSTM层的输出\n",
        "        lstm_out, hidden = self.lstm(embeds, hidden)\n",
        "        \n",
        "#         if bidirectional:\n",
        "#           lstm_out = lstm_out.contiguous().view(-1, self.hidden_dim*2)\n",
        "#         else:\n",
        "#           lstm_out = lstm_out.contiguous().view(-1, self.hidden_dim)\n",
        "       \n",
        "        # dropout与fully-connected层\n",
        "#         out = self.dropout(lstm_out)\n",
        "        out = lstm_out\n",
        "        out = out[:, -1]\n",
        "        out = self.fc(out)\n",
        "        # softmax \n",
        "#         soft_out = out\n",
        "        soft_out = self.soft(out)\n",
        "        \n",
        "        # reshape to be batch_size first\n",
        "#         soft_out = soft_out.view(batch_size, -1)\n",
        "#         soft_out = soft_out[:, -1] # get last batch of labels\n",
        "#         soft_out = np.where(soft_out==np.max(soft_out))\n",
        "    \n",
        "        return soft_out, hidden\n",
        "    \n",
        "    \n",
        "    def init_hidden(self, batch_size):\n",
        "\n",
        "        weight = next(self.parameters()).data\n",
        "        \n",
        "        number = 1\n",
        "        if self.bidirectional:\n",
        "           number = 2\n",
        "        \n",
        "        if (train_on_gpu):\n",
        "            hidden = (weight.new(self.n_layers*number, batch_size, self.hidden_dim).zero_().cuda(),\n",
        "                      weight.new(self.n_layers*number, batch_size, self.hidden_dim).zero_().cuda()\n",
        "                     )\n",
        "        else:\n",
        "            hidden = (weight.new(self.n_layers*number, batch_size, self.hidden_dim).zero_(),\n",
        "                      weight.new(self.n_layers*number, batch_size, self.hidden_dim).zero_()\n",
        "                     )\n",
        "        \n",
        "        return hidden"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": true,
        "id": "I5KtnthxnXKF",
        "colab_type": "code",
        "outputId": "9ab4c3a9-67ac-4ddc-976c-ca3ba7e512e8",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 143
        }
      },
      "source": [
        "# 设置超参数\n",
        "vocab_size = len(vocab_to_int)+1 # +1 for the 0 padding + our word tokens\n",
        "output_size = 3\n",
        "embedding_dim = 128\n",
        "hidden_dim = 196\n",
        "n_layers = 2\n",
        "bidirectional = False  #这里若为True，则是双向LSTM\n",
        " \n",
        "net = SentimentRNN(vocab_size, output_size, embedding_dim, hidden_dim, n_layers, bidirectional)\n",
        "\n",
        "print(net)"
      ],
      "execution_count": 28,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "SentimentRNN(\n",
            "  (embedding): Embedding(16723, 128)\n",
            "  (lstm): LSTM(128, 196, num_layers=2, batch_first=True, dropout=0.5)\n",
            "  (dropout): Dropout(p=0.3)\n",
            "  (fc): Linear(in_features=196, out_features=3, bias=True)\n",
            "  (soft): Softmax()\n",
            ")\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cYWHtkAbV3i3",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 917
        },
        "outputId": "033bf343-f81e-47ed-e194-be9310799cbc"
      },
      "source": [
        "output.squeeze()"
      ],
      "execution_count": 34,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[0.3335, 0.3447, 0.3218],\n",
              "        [0.3288, 0.3425, 0.3287],\n",
              "        [0.3234, 0.3482, 0.3284],\n",
              "        [0.3141, 0.3554, 0.3305],\n",
              "        [0.3210, 0.3442, 0.3348],\n",
              "        [0.3272, 0.3474, 0.3254],\n",
              "        [0.3171, 0.3506, 0.3323],\n",
              "        [0.3289, 0.3394, 0.3318],\n",
              "        [0.3260, 0.3419, 0.3321],\n",
              "        [0.3249, 0.3491, 0.3261],\n",
              "        [0.3175, 0.3451, 0.3373],\n",
              "        [0.3280, 0.3408, 0.3312],\n",
              "        [0.3269, 0.3457, 0.3275],\n",
              "        [0.3221, 0.3436, 0.3344],\n",
              "        [0.3242, 0.3465, 0.3293],\n",
              "        [0.3256, 0.3501, 0.3242],\n",
              "        [0.3203, 0.3422, 0.3376],\n",
              "        [0.3189, 0.3465, 0.3346],\n",
              "        [0.3205, 0.3464, 0.3331],\n",
              "        [0.3290, 0.3404, 0.3305],\n",
              "        [0.3255, 0.3429, 0.3317],\n",
              "        [0.3177, 0.3582, 0.3241],\n",
              "        [0.3191, 0.3514, 0.3294],\n",
              "        [0.3262, 0.3409, 0.3330],\n",
              "        [0.3252, 0.3509, 0.3239],\n",
              "        [0.3155, 0.3531, 0.3314],\n",
              "        [0.3238, 0.3513, 0.3249],\n",
              "        [0.3228, 0.3572, 0.3199],\n",
              "        [0.3234, 0.3381, 0.3385],\n",
              "        [0.3281, 0.3481, 0.3238],\n",
              "        [0.3271, 0.3502, 0.3227],\n",
              "        [0.3245, 0.3485, 0.3270],\n",
              "        [0.3198, 0.3507, 0.3295],\n",
              "        [0.3240, 0.3508, 0.3251],\n",
              "        [0.3240, 0.3360, 0.3400],\n",
              "        [0.3234, 0.3442, 0.3323],\n",
              "        [0.3175, 0.3496, 0.3329],\n",
              "        [0.3190, 0.3528, 0.3281],\n",
              "        [0.3188, 0.3632, 0.3179],\n",
              "        [0.3256, 0.3417, 0.3327],\n",
              "        [0.3277, 0.3506, 0.3218],\n",
              "        [0.3337, 0.3501, 0.3162],\n",
              "        [0.3246, 0.3463, 0.3290],\n",
              "        [0.3232, 0.3514, 0.3255],\n",
              "        [0.3239, 0.3528, 0.3232],\n",
              "        [0.3277, 0.3400, 0.3323],\n",
              "        [0.3256, 0.3470, 0.3274],\n",
              "        [0.3187, 0.3401, 0.3412],\n",
              "        [0.3313, 0.3462, 0.3225],\n",
              "        [0.3255, 0.3477, 0.3268]], device='cuda:0', grad_fn=<SqueezeBackward0>)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 34
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AWn3SZj5ut28",
        "colab_type": "code",
        "outputId": "66195335-d165-4f75-ea3f-53e27eee1997",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 917
        }
      },
      "source": [
        "output.long()"
      ],
      "execution_count": 46,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0],\n",
              "        [0, 0, 0]], device='cuda:0')"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 46
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mv2InWxWxvey",
        "colab_type": "code",
        "outputId": "e0dd2d11-e3e5-4512-e6c7-254233665e97",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 917
        }
      },
      "source": [
        "labels.long().float()"
      ],
      "execution_count": 42,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[1., 0., 0.],\n",
              "        [0., 0., 1.],\n",
              "        [0., 0., 1.],\n",
              "        [0., 0., 1.],\n",
              "        [1., 0., 0.],\n",
              "        [0., 1., 0.],\n",
              "        [0., 0., 1.],\n",
              "        [0., 0., 1.],\n",
              "        [0., 0., 1.],\n",
              "        [0., 1., 0.],\n",
              "        [0., 0., 1.],\n",
              "        [0., 1., 0.],\n",
              "        [1., 0., 0.],\n",
              "        [1., 0., 0.],\n",
              "        [0., 0., 1.],\n",
              "        [1., 0., 0.],\n",
              "        [0., 0., 1.],\n",
              "        [0., 0., 1.],\n",
              "        [0., 0., 1.],\n",
              "        [0., 0., 1.],\n",
              "        [0., 0., 1.],\n",
              "        [0., 1., 0.],\n",
              "        [1., 0., 0.],\n",
              "        [0., 1., 0.],\n",
              "        [1., 0., 0.],\n",
              "        [0., 0., 1.],\n",
              "        [0., 0., 1.],\n",
              "        [0., 1., 0.],\n",
              "        [0., 0., 1.],\n",
              "        [0., 0., 1.],\n",
              "        [0., 1., 0.],\n",
              "        [1., 0., 0.],\n",
              "        [0., 1., 0.],\n",
              "        [0., 0., 1.],\n",
              "        [1., 0., 0.],\n",
              "        [0., 0., 1.],\n",
              "        [1., 0., 0.],\n",
              "        [0., 0., 1.],\n",
              "        [0., 0., 1.],\n",
              "        [0., 0., 1.],\n",
              "        [0., 0., 1.],\n",
              "        [0., 0., 1.],\n",
              "        [0., 0., 1.],\n",
              "        [1., 0., 0.],\n",
              "        [1., 0., 0.],\n",
              "        [0., 0., 1.],\n",
              "        [1., 0., 0.],\n",
              "        [1., 0., 0.],\n",
              "        [0., 0., 1.],\n",
              "        [0., 0., 1.]], device='cuda:0')"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 42
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "deXMOK4qnXKJ",
        "colab_type": "text"
      },
      "source": [
        "开始训练："
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6yM13I4NnXKL",
        "colab_type": "code",
        "outputId": "6896b229-a596-4f27-ecac-b207ad5d55d5",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 433
        }
      },
      "source": [
        "# 使用交叉熵损失，设置学习率为0.001，使用Adam优化算法\n",
        "lr=0.001\n",
        " \n",
        "criterion = nn.CrossEntropyLoss()\n",
        "optimizer = torch.optim.Adam(net.parameters(), lr=lr)\n",
        " \n",
        "# 训练4轮 \n",
        "epochs = 4 \n",
        "# 每100步做一次输出 \n",
        "print_every = 100\n",
        "# 为了进行梯度裁剪，防止反向传播过程中出现梯度消失或者爆炸的情况\n",
        "clip=5 \n",
        " \n",
        "# 使用GPU\n",
        "if(train_on_gpu):\n",
        "    net.cuda()\n",
        "\n",
        "# 训练\n",
        "net.train()\n",
        "for e in range(epochs):\n",
        "    \n",
        "    # 隐层\n",
        "    h = net.init_hidden(batch_size)\n",
        "    counter = 0\n",
        " \n",
        "    # 按批量遍历\n",
        "    for inputs, labels in train_loader:\n",
        "        counter += 1\n",
        " \n",
        "        if(train_on_gpu):\n",
        "            inputs, labels = inputs.cuda(), labels.cuda()\n",
        " \n",
        "        h = tuple([each.data for each in h])\n",
        "        # 权值初始化\n",
        "        net.zero_grad()\n",
        "        # 得到模型输出\n",
        "        output, h = net(inputs, h)\n",
        "        \n",
        "        # 损失值\n",
        "        loss = criterion(output.squeeze(), labels.long().float())\n",
        "        # 反向传播\n",
        "        loss.backward()\n",
        "        # 做一个梯度裁剪\n",
        "        nn.utils.clip_grad_norm_(net.parameters(), clip)\n",
        "        # 更新权值\n",
        "        optimizer.step()\n",
        " \n",
        "        if counter % print_every == 0:\n",
        "            # 获得验证集loss，与训练集过程类似\n",
        "            val_h = net.init_hidden(batch_size)\n",
        "            val_losses = []\n",
        "            net.eval()\n",
        "            for inputs, labels in valid_loader:\n",
        " \n",
        "                val_h = tuple([each.data for each in val_h])\n",
        " \n",
        "                if(train_on_gpu):\n",
        "                    inputs, labels = inputs.cuda(), labels.cuda()\n",
        " \n",
        "                output, val_h = net(inputs, val_h)\n",
        "                val_loss = criterion(output.squeeze(), labels.long().float())\n",
        " \n",
        "                val_losses.append(val_loss.item())\n",
        " \n",
        "            net.train()\n",
        "            print(\"Epoch: {}/{}...\".format(e+1, epochs),\n",
        "                  \"Step: {}...\".format(counter),\n",
        "                  \"训练损失: {:.6f}...\".format(loss.item()),\n",
        "                  \"验证损失: {:.6f}\".format(np.mean(val_losses)))"
      ],
      "execution_count": 43,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:57: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "error",
          "ename": "RuntimeError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mRuntimeError\u001b[0m                              Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-43-0520727d0d7c>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m     37\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     38\u001b[0m         \u001b[0;31m# 损失值\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 39\u001b[0;31m         \u001b[0mloss\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcriterion\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0moutput\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msqueeze\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlabels\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlong\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfloat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     40\u001b[0m         \u001b[0;31m# 反向传播\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     41\u001b[0m         \u001b[0mloss\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbackward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, *input, **kwargs)\u001b[0m\n\u001b[1;32m    491\u001b[0m             \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_slow_forward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    492\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 493\u001b[0;31m             \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mforward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    494\u001b[0m         \u001b[0;32mfor\u001b[0m \u001b[0mhook\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_forward_hooks\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mvalues\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    495\u001b[0m             \u001b[0mhook_result\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mhook\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mresult\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/torch/nn/modules/loss.py\u001b[0m in \u001b[0;36mforward\u001b[0;34m(self, input, target)\u001b[0m\n\u001b[1;32m    940\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mforward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtarget\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    941\u001b[0m         return F.cross_entropy(input, target, weight=self.weight,\n\u001b[0;32m--> 942\u001b[0;31m                                ignore_index=self.ignore_index, reduction=self.reduction)\n\u001b[0m\u001b[1;32m    943\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    944\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py\u001b[0m in \u001b[0;36mcross_entropy\u001b[0;34m(input, target, weight, size_average, ignore_index, reduce, reduction)\u001b[0m\n\u001b[1;32m   2054\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0msize_average\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0mreduce\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2055\u001b[0m         \u001b[0mreduction\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_Reduction\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlegacy_get_string\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msize_average\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mreduce\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2056\u001b[0;31m     \u001b[0;32mreturn\u001b[0m \u001b[0mnll_loss\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlog_softmax\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtarget\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mweight\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mignore_index\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mreduction\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2057\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2058\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py\u001b[0m in \u001b[0;36mnll_loss\u001b[0;34m(input, target, weight, size_average, ignore_index, reduce, reduction)\u001b[0m\n\u001b[1;32m   1869\u001b[0m                          .format(input.size(0), target.size(0)))\n\u001b[1;32m   1870\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mdim\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1871\u001b[0;31m         \u001b[0mret\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_C\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_nn\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnll_loss\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtarget\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mweight\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0m_Reduction\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_enum\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mreduction\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mignore_index\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1872\u001b[0m     \u001b[0;32melif\u001b[0m \u001b[0mdim\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m4\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1873\u001b[0m         \u001b[0mret\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_C\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_nn\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnll_loss2d\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtarget\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mweight\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0m_Reduction\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_enum\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mreduction\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mignore_index\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mRuntimeError\u001b[0m: Expected object of scalar type Long but got scalar type Float for argument #2 'target'"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Dxn55y9bnXKQ",
        "colab_type": "text"
      },
      "source": [
        "开始测试："
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZCiuKyxinXKS",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "test_losses = [] \n",
        "num_correct = 0\n",
        " \n",
        "# 隐层\n",
        "h = net.init_hidden(batch_size)\n",
        " \n",
        "net.eval()\n",
        "# 将测试数据进行迭代遍历\n",
        "for inputs, labels in test_loader:\n",
        " \n",
        "    h = tuple([each.data for each in h])\n",
        " \n",
        "    if(train_on_gpu):\n",
        "        inputs, labels = inputs.cuda(), labels.cuda()\n",
        "    \n",
        "    # 得到预测输出\n",
        "    output, h = net(inputs, h)\n",
        "    \n",
        "    # 测试损失\n",
        "    test_loss = criterion(output.squeeze(), labels.float())\n",
        "    test_losses.append(test_loss.item())\n",
        "    \n",
        "#     pred = torch.round(output.squeeze())  \n",
        "    \n",
        "    # compare predictions to true label\n",
        "#     correct_tensor = pred.eq(labels.float().view_as(pred))\n",
        "#     correct = np.squeeze(correct_tensor.numpy()) if not train_on_gpu else np.squeeze(correct_tensor.cpu().numpy())\n",
        "#     num_correct += np.sum(correct)\n",
        " \n",
        " \n",
        "# avg test loss\n",
        "print(\"Test loss: {:.3f}\".format(np.mean(test_losses)))\n",
        " \n",
        "# accuracy over all test data\n",
        "# test_acc = num_correct/len(test_loader.dataset)\n",
        "# print(\"Test accuracy: {:.3f}\".format(test_acc))f"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FFdLLRWvnXKW",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UM7B6rV2nXKe",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vj_QkmBAnXKi",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CGcfJHzOnXKm",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YIQ2vEAGnXKs",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "o3MaK1RanXKy",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "x4AE8WrjnXK2",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7AjFOAICnXK7",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UDuGHiwtnXK-",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lKo0I-pPnXLC",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}