Advanced_tensorflow_Mnist.ipynb

{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": 114,
      "metadata": {
        "id": "ZqRZpK5YAMkd"
      },
      "outputs": [],
      "source": [
        "import tensorflow as tf\n",
        "from tensorflow import keras\n",
        "from keras.layers import Dense , Flatten , Conv2D , MaxPooling2D"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "v5B4btiHAMke"
      },
      "source": [
        "## Tensorflow for experts\n",
        "its begginer version , often used for simple or small networks and tasks .<br>\n",
        "like  simple classification , regression ...<br>\n",
        "\n",
        "but for huge models like chatgpt , and real world problems , we need to do all tasks ourselves , like compiling , training , evaluating .\n",
        "all of these steps we have used were automatic : <br>\n",
        "model.compile() <br>\n",
        "model.fit() <br>\n",
        "model.evaluate() <br>\n",
        "<br>\n",
        "but now we are going to do all of these ourselves"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "81Rm1MxYAMkf"
      },
      "source": [
        "### Train MNIST Dataset , with this more difficult way :"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# steps :\n",
        "0_ load data <br>\n",
        "1_ normalize data (pixels) <br>\n",
        "2_ check numbers of channels (in case of gray scale image , unsqueeze X data.\n",
        "<br>\n",
        "4_ Adding batch size , shuffle , and etc.\n"
      ],
      "metadata": {
        "id": "76bUgOAmAw8o"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 115,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 68
        },
        "id": "0GuNTE2JAMkg",
        "outputId": "f9484f82-81b3-4d3f-df0c-63176e28e9b7"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "         22, 137, 243, 255, 173,   2,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  20,\n",
              "        185, 202,  87,  90, 254,   6,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  31, 214,\n",
              "        160,   8,   0,  70, 254,   6,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0, 202, 185,\n",
              "          3,   0,   0,  77, 232,   4,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  97, 250,  22,\n",
              "          0,   0,   1, 177, 138,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,  23, 227, 146,   0,\n",
              "          0,   0,  15, 254,  76,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,  73, 254,  53,   0,\n",
              "          0,   6, 191, 254,  63,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,  73, 254,  17,   0,\n",
              "         61, 200, 254, 252,  34,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,  44, 244, 196, 158,\n",
              "        244, 164, 225, 206,  17,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  51, 185, 221,\n",
              "        152,  18, 211, 184,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,  21, 237, 104,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0, 128, 246,  23,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0, 155, 240,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0, 155, 174,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0, 168, 161,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "         14, 235,  99,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "         59, 254,  43,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "        100, 246,  29,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "        120, 194,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "        165, 114,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0]], dtype=uint8)"
            ],
            "text/html": [
              "<style>\n",
              "      .ndarray_repr .ndarray_raw_data {\n",
              "        display: none;\n",
              "      }\n",
              "      .ndarray_repr.show_array .ndarray_raw_data {\n",
              "        display: block;\n",
              "      }\n",
              "      .ndarray_repr.show_array .ndarray_image_preview {\n",
              "        display: none;\n",
              "      }\n",
              "      </style>\n",
              "      <div id=\"id-61f74e50-633b-4520-9057-c8667fa40e7e\" class=\"ndarray_repr\"><pre>ndarray (28, 28) <button style=\"padding: 0 2px;\">show data</button></pre><img src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAAAAABXZoBIAAAAsElEQVR4nGNgGLpArPPz/7VM2OVEdp4Kj/rHhlVO/toCDgY3HJKndjIzMPi+YMEml/hLjIGBcWMXNjnxx5MYGBj4//lgk/T8Z8rAwLb/nz12SUEG2xP//ighhBC2P/62+VbYBsm377E61njn3RlCl3dglWNgYGBgEH2bgVuy4Zs4Eg8tIKV/vcQtybCPAbck72fcVvK9TsYtaf1PG7exmj8+4ZYUOfMYt6TuBtxWkgQAmlIyUa6aT8sAAAAASUVORK5CYII=\" class=\"ndarray_image_preview\" /><pre class=\"ndarray_raw_data\">array([[  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "         22, 137, 243, 255, 173,   2,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  20,\n",
              "        185, 202,  87,  90, 254,   6,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  31, 214,\n",
              "        160,   8,   0,  70, 254,   6,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0, 202, 185,\n",
              "          3,   0,   0,  77, 232,   4,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  97, 250,  22,\n",
              "          0,   0,   1, 177, 138,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,  23, 227, 146,   0,\n",
              "          0,   0,  15, 254,  76,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,  73, 254,  53,   0,\n",
              "          0,   6, 191, 254,  63,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,  73, 254,  17,   0,\n",
              "         61, 200, 254, 252,  34,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,  44, 244, 196, 158,\n",
              "        244, 164, 225, 206,  17,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  51, 185, 221,\n",
              "        152,  18, 211, 184,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,  21, 237, 104,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0, 128, 246,  23,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0, 155, 240,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0, 155, 174,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0, 168, 161,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "         14, 235,  99,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "         59, 254,  43,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "        100, 246,  29,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "        120, 194,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "        165, 114,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0]], dtype=uint8)</pre></div><script>\n",
              "      (() => {\n",
              "      const titles = ['show data', 'hide data'];\n",
              "      let index = 0\n",
              "      document.querySelector('#id-61f74e50-633b-4520-9057-c8667fa40e7e button').onclick = (e) => {\n",
              "        document.querySelector('#id-61f74e50-633b-4520-9057-c8667fa40e7e').classList.toggle('show_array');\n",
              "        index = (++index) % 2;\n",
              "        document.querySelector('#id-61f74e50-633b-4520-9057-c8667fa40e7e button').textContent = titles[index];\n",
              "        e.preventDefault();\n",
              "        e.stopPropagation();\n",
              "      }\n",
              "      })();\n",
              "    </script>"
            ]
          },
          "metadata": {},
          "execution_count": 115
        }
      ],
      "source": [
        "MNIST = keras.datasets.mnist\n",
        "\n",
        "# تابع لود دیتا ۴ تا مقدار برمیگردونه یعنی دو تا تاپل\n",
        "#Returns:\n",
        "#  Tuple of NumPy arrays: (x_train, y_train), (x_test, y_test).\n",
        "(x_train , y_train) , (x_test , y_test) = MNIST.load_data()\n",
        "\n",
        "x_train[900]"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# normalizing pixels :\n",
        "x_train , x_test = x_train/255.0 , x_test/255.0 # should be float"
      ],
      "metadata": {
        "id": "ihCbJ2ElArPY"
      },
      "execution_count": 116,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "x_train[5].shape"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Q-mEqS07L9wM",
        "outputId": "a20acd4d-8264-4bc6-8cbd-8b34489e42e7"
      },
      "execution_count": 117,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(28, 28)"
            ]
          },
          "metadata": {},
          "execution_count": 117
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "because we want to use 2D conv , so we have to convert (28,28) image , into a 3 channel image : <br>\n",
        "(28,28,1)\n",
        "<br>\n",
        "* ps : in other datasets we dont have this problem . for example , in CIFAR10 , images have 3 channels their selves (32,32,3) .\n"
      ],
      "metadata": {
        "id": "lTgfWCSDMZnf"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# or\n",
        "#x_train = x_train.reshape()\n",
        "\n",
        "#or\n",
        "#x_train = tf.expand_dims(x_train , axis=3)\n",
        "#x_train[5].shape"
      ],
      "metadata": {
        "id": "oaaJ2U45RplA"
      },
      "execution_count": 118,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# add channel layer - << unsqueeze >>\n",
        "x_train = x_train[... , tf.newaxis]\n",
        "x_test = x_test[... , tf.newaxis]\n",
        "x_train[5].shape"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "G30rgxzKMZTB",
        "outputId": "9344c96b-67d3-4585-8e90-9c9d587872fb"
      },
      "execution_count": 119,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(28, 28, 1)"
            ]
          },
          "metadata": {},
          "execution_count": 119
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "type(x_train[5][0,0,0])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "EHxFmXbnWnY7",
        "outputId": "cecbd189-450a-41e3-a426-5cc7214ec21e"
      },
      "execution_count": 120,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "numpy.float64"
            ]
          },
          "metadata": {},
          "execution_count": 120
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Adding batch size :"
      ],
      "metadata": {
        "id": "Fp46-W8qYdwo"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "train_dataset = tf.data.Dataset.from_tensor_slices((x_train , y_train)).batch(32).shuffle(buffer_size=1000)\n",
        "test_dataset = tf.data.Dataset.from_tensor_slices((x_test , y_test)).batch(32)\n",
        "\n",
        "# we only need to shuffle train data"
      ],
      "metadata": {
        "id": "e7pM5S9MXNH6"
      },
      "execution_count": 121,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## creating model using object oriented"
      ],
      "metadata": {
        "id": "8it11ruRakn_"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "class MyModel(tf.keras.Model):\n",
        "  def __init__(self):\n",
        "    super().__init__()\n",
        "\n",
        "    # define network layers\n",
        "    self.conv2D_1  = Conv2D(32 ,3 , activation=\"relu\") # 32 filter with size = 3\n",
        "    self.conv2D_2  = Conv2D(64 ,3 , activation=\"relu\")\n",
        "    self.maxpool2D = MaxPooling2D()\n",
        "    self.flatten = Flatten()\n",
        "    self.dense1 = Dense(128 , activation=\"relu\")\n",
        "    self.dense2 = Dense(10)\n",
        "\n",
        "\n",
        "  def call(self , x):\n",
        "    x = self.conv2D_1(x)\n",
        "    x = self.maxpool2D(x)\n",
        "    x = self.conv2D_2(x)\n",
        "    x = self.maxpool2D(x)\n",
        "    x = self.flatten(x)\n",
        "    x = self.dense1(x)\n",
        "    x = self.dense2(x)\n",
        "    return x\n",
        "\n",
        "\n",
        "\n",
        "model = MyModel()"
      ],
      "metadata": {
        "id": "vDfEmo2uXL5i"
      },
      "execution_count": 122,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## implement compile and fit methods :"
      ],
      "metadata": {
        "id": "EGCNoAd9j4MJ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# REAL / MAIN loss-function uses in train stage -\n",
        "# which affect on updating network's weights\n",
        "loss_function = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)\n",
        "\n",
        "optimizer = tf.keras.optimizers.Adam()\n"
      ],
      "metadata": {
        "id": "Kh65zoRUkDkK"
      },
      "execution_count": 123,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# these are METRICS\n",
        "# they are not going to use in train process\n",
        "\n",
        "# another loss which we use in evaluation\n",
        "train_loss = tf.keras.metrics.Mean()\n",
        "test_loss = tf.keras.metrics.Mean()\n",
        "\n",
        "# bc problem is classification , so we also need accuracy\n",
        "train_accuracy = tf.keras.metrics.SparseCategoricalAccuracy()\n",
        "test_accuracy = tf.keras.metrics.SparseCategoricalAccuracy()"
      ],
      "metadata": {
        "id": "wVHX5bXI2Jyh"
      },
      "execution_count": 124,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# TRAIN"
      ],
      "metadata": {
        "id": "V-l0Mkyn8VfB"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## important tip ::\n",
        "## we should RESET VALUES of : <br>\n",
        "### train loss , train acc , test loss , test acc , IN EACH EPOCH"
      ],
      "metadata": {
        "id": "QQ6K_MbHGSFw"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "چون داخل خود مدل وزن ها داره آپدیت میشه و مدل یک متغییر گلوبال هست نیازی نیست در توابع ترین و تست چیزی ریترن بشه"
      ],
      "metadata": {
        "id": "nYX-feSuItUV"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "@tf.function : <br>\n",
        "please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing."
      ],
      "metadata": {
        "id": "F7uXwBSgJShF"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "@tf.function\n",
        "def train(images , labels):\n",
        "    with tf.GradientTape() as tape : # compute gradient\n",
        "      # 1_ give data to network :\n",
        "      prediction = model(images)\n",
        "      # 2_ compare these preddictions with real labels & compute loss :\n",
        "      loss = loss_function(labels , prediction)\n",
        "    # 3_ compute GRADIENTS\n",
        "    gradients = tape.gradient(loss , model.trainable_variables) # trainable netwok weights\n",
        "\n",
        "    # 4_update weights :\n",
        "    optimizer.apply_gradients(grads_and_vars= zip(gradients , model.trainable_variables))\n",
        "\n",
        "    train_loss(loss)\n",
        "    train_accuracy(labels , prediction)\n"
      ],
      "metadata": {
        "id": "Haqf23ZKHn7z"
      },
      "execution_count": 125,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "@tf.function\n",
        "def test(images , labels):\n",
        "  # it looks like train stage , but without computing gradients\n",
        "  # ps : in test phase we dont need to compute moshtagh(gradients)\n",
        "  prediction = model(images)\n",
        "  loss = loss_function(y_true=labels , y_pred=prediction)\n",
        "  test_loss(loss)\n",
        "  test_accuracy(labels , prediction)\n",
        "\n"
      ],
      "metadata": {
        "id": "T6UzpnriIAEf"
      },
      "execution_count": 126,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "epochs = 8\n",
        "for epoch in range(epochs):\n",
        "  train_loss.reset_states()\n",
        "  train_accuracy.reset_states()\n",
        "  test_loss.reset_states()\n",
        "  test_accuracy.reset_states()\n",
        "\n",
        "  #train\n",
        "  for images , labels in train_dataset :\n",
        "    train(images , labels)\n",
        "\n",
        "  # test\n",
        "  for images , labels in test_dataset :\n",
        "    test(images , labels)\n",
        "\n",
        "\n",
        "  print(\"epoch:\" , epoch + 1  ,\n",
        "        f\"Train Loss: {train_loss.result()}\" ,\n",
        "        f\"Train Acc: {train_accuracy.result()}\" ,\n",
        "        f\"Test loss: {test_loss.result()}\",\n",
        "        f\"Test Acc: {test_accuracy.result()}\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "T6XRnkv53eLT",
        "outputId": "0bfed112-bc18-43bb-bdb0-efb48f834a65"
      },
      "execution_count": 127,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "epoch: 1 Train Loss: 0.1338050365447998 Train Acc: 0.9590333104133606 Test loss: 0.0361843965947628 Test Acc: 0.9876999855041504\n",
            "epoch: 2 Train Loss: 0.04314655065536499 Train Acc: 0.9863666892051697 Test loss: 0.036130405962467194 Test Acc: 0.9886000156402588\n",
            "epoch: 3 Train Loss: 0.0305416751652956 Train Acc: 0.9898333549499512 Test loss: 0.03035852499306202 Test Acc: 0.9896000027656555\n",
            "epoch: 4 Train Loss: 0.020896289497613907 Train Acc: 0.9935500025749207 Test loss: 0.02825453318655491 Test Acc: 0.9908000230789185\n",
            "epoch: 5 Train Loss: 0.016112733632326126 Train Acc: 0.9947166442871094 Test loss: 0.03157404065132141 Test Acc: 0.9902999997138977\n",
            "epoch: 6 Train Loss: 0.0115890521556139 Train Acc: 0.9960166811943054 Test loss: 0.036296993494033813 Test Acc: 0.9907000064849854\n",
            "epoch: 7 Train Loss: 0.010043875314295292 Train Acc: 0.9966999888420105 Test loss: 0.03119869902729988 Test Acc: 0.9894000291824341\n",
            "epoch: 8 Train Loss: 0.007445247378200293 Train Acc: 0.9975000023841858 Test loss: 0.02790781296789646 Test Acc: 0.9933000206947327\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "model.save(\"advanced_tf_model_mnist\")"
      ],
      "metadata": {
        "id": "adZo5G6bPSZE"
      },
      "execution_count": 128,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# INFERENCE stage :\n",
        "TEST an iamge"
      ],
      "metadata": {
        "id": "45VppuU9K_Tm"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "model = tf.keras.models.load_model(\"advanced_tf_model_mnist\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ozD8nHuIP5L8",
        "outputId": "e24febe7-c5e4-4a52-8e21-7b96564d8e7f"
      },
      "execution_count": 129,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "WARNING:tensorflow:No training configuration found in save file, so the model was *not* compiled. Compile it manually.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import cv2\n",
        "import numpy as np\n",
        "\n",
        "image =  cv2.imread(\"seven.png\")\n",
        "image = cv2.cvtColor(image , cv2.COLOR_BGR2GRAY)\n",
        "image = cv2.resize(image , (28,28))\n",
        "image = image[...,tf.newaxis]\n",
        "image = image.astype(\"float32\") # bc new image is integer so it should be converted to float\n",
        "image.shape\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "b3hHUNbO3eH0",
        "outputId": "f8a6b659-bf53-4bfe-b7b9-a681369e7e50"
      },
      "execution_count": 130,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(28, 28, 1)"
            ]
          },
          "metadata": {},
          "execution_count": 130
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# data should enter the network in batch shape\n",
        "# for example :\n",
        "# if  batch size = 32\n",
        "# then : we have (32 , 28 , 28 , 1)\n",
        "\n",
        "# but now be have only 1 image for test , so :\n",
        "# (1 , 28 , 28 , 1 )\n",
        "\n",
        "image  = image[tf.newaxis , ...]\n",
        "image.shape\n",
        "#(batchsize , width , height , img_channel_number)\n",
        "# if we had colored image , img_channel_number became 3"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kNLO-3Gd3eFC",
        "outputId": "77da3ed1-afb9-4352-d8ce-e588d271be4a"
      },
      "execution_count": 131,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(1, 28, 28, 1)"
            ]
          },
          "metadata": {},
          "execution_count": 131
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# so network likes to get a 4D input\n",
        "(batchsize , width , height , img_channel_number)"
      ],
      "metadata": {
        "id": "jkzbDEysNwde"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "prediction = model(image)\n",
        "prediction"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "euZ_EYQ3NH2a",
        "outputId": "d96c6334-f77c-42fa-a817-f785796d58e6"
      },
      "execution_count": 132,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(1, 10), dtype=float32, numpy=\n",
              "array([[-3872.9617 , -2256.079  ,  -335.79745,   839.97314,    26.22779,\n",
              "        -1813.1091 , -5181.0127 ,  6131.1865 , -1705.3674 , -1412.0598 ]],\n",
              "      dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 132
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "np.argmax(prediction)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hWBPcHzeO6hT",
        "outputId": "da709fec-3fba-4a54-e58c-57c5dd30dd8c"
      },
      "execution_count": 133,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "7"
            ]
          },
          "metadata": {},
          "execution_count": 133
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## only this loss : loss='sparse_categorical_crossentropy'\n",
        "is attending in train process . <br>\n",
        "another losses like mae and mse dont attend in training process and dont affects network weights."
      ],
      "metadata": {
        "id": "Js1VqbIh4Sds"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# for beginners version\n",
        "import tensorflow as tf\n",
        "mnist = tf.keras.datasets.mnist\n",
        "\n",
        "(x_train, y_train),(x_test, y_test) = mnist.load_data()\n",
        "x_train, x_test = x_train / 255.0, x_test / 255.0\n",
        "\n",
        "model = tf.keras.models.Sequential([\n",
        "  tf.keras.layers.Flatten(input_shape=(28, 28)),\n",
        "  tf.keras.layers.Dense(128, activation='relu'),\n",
        "  tf.keras.layers.Dropout(0.2),\n",
        "  tf.keras.layers.Dense(10, activation='softmax')\n",
        "])\n",
        "\n",
        "model.compile(optimizer='adam',\n",
        "              loss='sparse_categorical_crossentropy', # use this loss for train , and updating weights\n",
        "              metrics=['accuracy' , \"mae\" , \"mse\"])\n",
        "\n",
        "\n",
        "model.fit(x_train, y_train,validation_split=0.2, epochs=8)\n",
        "model.evaluate(x_test, y_test)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-gve3pTh3d_o",
        "outputId": "a0d40798-cd76-4dec-a72e-99cbc458a947"
      },
      "execution_count": 134,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/8\n",
            "1500/1500 [==============================] - 8s 4ms/step - loss: 0.3284 - accuracy: 0.9048 - val_loss: 0.1662 - val_accuracy: 0.9522\n",
            "Epoch 2/8\n",
            "1500/1500 [==============================] - 6s 4ms/step - loss: 0.1566 - accuracy: 0.9540 - val_loss: 0.1178 - val_accuracy: 0.9654\n",
            "Epoch 3/8\n",
            "1500/1500 [==============================] - 5s 3ms/step - loss: 0.1169 - accuracy: 0.9647 - val_loss: 0.0993 - val_accuracy: 0.9711\n",
            "Epoch 4/8\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.0991 - accuracy: 0.9702 - val_loss: 0.1000 - val_accuracy: 0.9696\n",
            "Epoch 5/8\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.0810 - accuracy: 0.9743 - val_loss: 0.0938 - val_accuracy: 0.9716\n",
            "Epoch 6/8\n",
            "1500/1500 [==============================] - 5s 3ms/step - loss: 0.0701 - accuracy: 0.9777 - val_loss: 0.0880 - val_accuracy: 0.9742\n",
            "Epoch 7/8\n",
            "1500/1500 [==============================] - 4s 3ms/step - loss: 0.0617 - accuracy: 0.9802 - val_loss: 0.0837 - val_accuracy: 0.9748\n",
            "Epoch 8/8\n",
            "1500/1500 [==============================] - 5s 4ms/step - loss: 0.0551 - accuracy: 0.9824 - val_loss: 0.0804 - val_accuracy: 0.9785\n",
            "313/313 [==============================] - 1s 2ms/step - loss: 0.0679 - accuracy: 0.9794\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[0.06786849349737167, 0.9793999791145325]"
            ]
          },
          "metadata": {},
          "execution_count": 134
        }
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python",
      "version": "3.10.1"
    },
    "colab": {
      "provenance": [],
      "gpuType": "T4"
    },
    "accelerator": "GPU"
  },
  "nbformat": 4,
  "nbformat_minor": 0
}