5.1-Review-Autoencoders.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a href=\"https://www.bigdatauniversity.com\"><img src=\"https://ibm.box.com/shared/static/qo20b88v1hbjztubt06609ovs85q8fau.png\" width=\"400px\" align=\"center\"></a>\n",
    "\n",
    "<h1 align=\"center\"><font size=\"5\">AUTOENCODERS</font></h1>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-info\" style=\"margin-top: 20px\">\n",
    "Welcome to this notebook about autoencoders.\n",
    "<font size=\"3\"><strong>In this notebook you will find an explanation of what is an autoencoder, how it works, and see an implementation of an autoencoder in TensorFlow.</strong></font>\n",
    "<br>\n",
    "<br>\n",
    "<h2>Table of Contents</h2>\n",
    "<ol>\n",
    " <li><a href=\"#ref1\">Introduction</a></li>\n",
    " <li><a href=\"#ref2\">Feature Extraction and Dimensionality Reduction</a></li>\n",
    " <li><a href=\"#ref3\">Autoencoder Structure</a></li>\n",
    " <li><a href=\"#ref4\">Performance</a></li>\n",
    " <li><a href=\"#ref5\">Training: Loss Function</a></li>\n",
    " <li><a href=\"#ref6\">Code</a></li>\n",
    "</ol>\n",
    "</div>\n",
    "<br>\n",
    "By the end of this notebook, you should be able to create simple autoencoders and how to apply them to problems that involves unsupervised learning.\n",
    "<br>\n",
    "<p></p>\n",
    "<hr>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"ref1\"></a>\n",
    "<h2>Introduction</h2>\n",
    "An autoencoder, also known as autoassociator or Diabolo networks, is an artificial neural network employed to recreate the given input.\n",
    "It takes a set of <b>unlabeled</b> inputs, encodes them and then tries to extract the most valuable information from them.\n",
    "They are used for feature extraction, learning generative models of data, dimensionality reduction and can be used for compression. \n",
    "\n",
    "A 2006 paper named <b><a href=\"https://www.cs.toronto.edu/~hinton/science.pdf\">Reducing the Dimensionality of Data with Neural Networks</a>, done by G. E. Hinton and R. R. Salakhutdinov</b>, showed better results than years of refining other types of network, and was a breakthrough in the field of Neural Networks, a field that was \"stagnant\" for 10 years.\n",
    "\n",
    "Now, autoencoders, based on Restricted Boltzmann Machines, are employed in some of the largest deep learning applications. They are the building blocks of Deep Belief Networks (DBN).\n",
    "\n",
    "<center><img src=\"https://ibm.box.com/shared/static/xlkv9v7xzxhjww681dq3h1pydxcm4ktp.png\" style=\"width: 350px;\"></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"ref2\"></a>\n",
    "<h2>Feature Extraction and Dimensionality Reduction</h2>\n",
    "\n",
    "An example given by Nikhil Buduma in KdNuggets (<a href=\"http://www.kdnuggets.com/2015/03/deep-learning-curse-dimensionality-autoencoders.html\">link</a>) which gave an excellent explanation of the utility of this type of Neural Network.\n",
    "\n",
    "Say that you want to extract what emotion the person in a photography is feeling. Using the following 256x256 pixel grayscale picture as an example:\n",
    "\n",
    "<img src=\"https://ibm.box.com/shared/static/r5knpow4bk2farlvxia71e9jp2f2u126.png\">\n",
    "\n",
    "But when use this picture we start running into a bottleneck! Because this image being 256x256 pixels in size correspond with an input vector of 65536 dimensions! If we used an image produced with conventional cellphone cameras, that generates images of 4000 x 3000 pixels, we would have 12 million dimensions to analyze.\n",
    "\n",
    "\n",
    "This bottleneck is further problematized as the difficulty of a machine learning problem is increased as more dimensions are involved. According to a 1982 study by C.J. Stone (<a href=\"http://www-personal.umich.edu/~jizhu/jizhu/wuke/Stone-AoS82.pdf\">link</a>), the time to fit a model, is optimal if:\n",
    "\n",
    "<br><br>\n",
    "<div class=\"alert alert-block alert-info\" style=\"margin-top: 20px\">\n",
    "<h3><strong>$$m^{-p/(2p+d)}$$</strong></h3>\n",
    "<br>\n",
    "Where:\n",
    "<br>\n",
    "m: Number of data points\n",
    "<br>\n",
    "d: Dimensionality of the data\n",
    "<br>\n",
    "p: Parameter that depends on the model\n",
    "</div>\n",
    "\n",
    "As you can see, it increases exponentially!\n",
    "Returning to our example, we don't need to use all of the 65,536 dimensions to classify an emotion. A human identify emotions according to some specific facial expression, some <b>key features</b>, like the shape of the mouth and eyebrows.\n",
    "\n",
    "<center><img src=\"https://ibm.box.com/shared/static/m8urvuqujkt2vt1ru1fnslzh24pv7hn4.png\" height=\"256\" width=\"256\"></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"ref3\"></a>\n",
    "<h2>Autoencoder Structure</h2>\n",
    "\n",
    "<img src=\"https://ibm.box.com/shared/static/no7omt2jhqvv7uuls7ihnzikyl9ysnfp.png\" style=\"width: 400px;\">\n",
    "\n",
    "An autoencoder can be divided in two parts, the <b>encoder</b> and the <b>decoder</b>.\n",
    "\n",
    "The encoder needs to compress the representation of an input. In this case we are going to reduce the dimension the face of our actor, from 2000 dimensions to only 30 dimensions, by running the data through layers of our encoder.\n",
    "\n",
    "The decoder works like encoder network in reverse. It works to recreate the input, as closely as possible. This plays an important role during training, because it forces the autoencoder to select the most important features in the compressed representation.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"ref4\"></a>\n",
    "<h2>Performance</h2>\n",
    "\n",
    "After the training has been done, you can use the encoded data as a reliable dimensionally-reduced data, applying it to any problems where dimensionality reduction seems appropriate.\n",
    "\n",
    "<img src=\"https://ibm.box.com/shared/static/yt3xyon4g2jyw1w9qup1mvx7cgh28l64.png\">\n",
    "\n",
    "This image was extracted from the G. E. Hinton and R. R. Salakhutdinovcomparing's <a href=\"https://www.cs.toronto.edu/~hinton/science.pdf\">paper</a>, on the two-dimensional reduction for 500 digits of the MNIST, with PCA on the left and autoencoder on the right. We can see that the autoencoder provided us with a better separation of data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"ref5\"></a>\n",
    "<h2>Training: Loss function</h2>\n",
    "\n",
    "An autoencoder uses the Loss function to properly train the network. The Loss function will calculate the differences between our output and the expected results. After that, we can minimize this error with gradient descent. There are more than one type of Loss function, it depends on the type of data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3>Binary Values:</h3>\n",
    "$$l(f(x)) = - \\sum_{k} (x_k log(\\hat{x}_k) + (1 - x_k) \\log (1 - \\hat{x}_k) \\ )$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For binary values, we can use an equation based on the sum of Bernoulli's cross-entropy. \n",
    "\n",
    "$x_k$ is one of our inputs and $\\hat{x}_k$ is the respective output.\n",
    "\n",
    "We use this function so that if $x_k$ equals to one, we want to push $\\hat{x}_k$ as close as possible to one. The same if $x_k$ equals to zero.\n",
    "\n",
    "If the value is one, we just need to calculate the first part of the formula, that is, $- x_k log(\\hat{x}_k)$. Which, turns out to just calculate $- log(\\hat{x}_k)$.\n",
    "\n",
    "And if the value is zero, we need to calculate just the second part, $(1 - x_k) \\log (1 - \\hat{x}_k) \\ )$ - which turns out to be $log (1 - \\hat{x}_k) $.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3>Real values:</h3>\n",
    "$$l(f(x)) = - \\frac{1}{2}\\sum_{k} (\\hat{x}_k- x_k \\ )^2$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As the above function would behave badly with inputs that are not 0 or 1, we can use the sum of squared differences for our Loss function. If you use this loss function, it's necessary that you use a linear activation function for the output layer.\n",
    "\n",
    "As it was with the above example, $x_k$ is one of our inputs and $\\hat{x}_k$ is the respective output, and we want to make our output as similar as possible to our input."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3>Loss Gradient:</h3>\n",
    "\n",
    "$$\\nabla_{\\hat{a}(x^{(t)})} \\ l( \\ f(x^{(t)}))  = \\hat{x}^{(t)} - x^{(t)} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We use the gradient descent to reach the local minimum of our function $l( \\ f(x^{(t)})$, taking steps towards the negative of the gradient of the function in the current point.\n",
    "\n",
    "Our function about the gradient $(\\nabla_{\\hat{a}(x^{(t)})})$ of the loss of $l( \\ f(x^{(t)})$ in the preactivation of the output layer.\n",
    "\n",
    "It's actually a simple formula, it is done by calculating the difference between our output $\\hat{x}^{(t)}$ and our input $x^{(t)}$.\n",
    "\n",
    "Then our network backpropagates our gradient $\\nabla_{\\hat{a}(x^{(t)})} \\ l( \\ f(x^{(t)}))$ through the network using <b>backpropagation</b>."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"ref6\"></a>\n",
    "<h2>Code</h2>\n",
    "\n",
    "For this part, we walk through a lot of Python 2.7.11 code. We are going to use the MNIST dataset for our example.\n",
    "The following code was created by Aymeric Damien. You can find some of his code in <a href=\"https://github.com/aymericdamien\">here</a>. We made some modifications for us to import the datasets to Jupyter Notebooks."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's call our imports and make the MNIST data available to use."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: Logging before flag parsing goes to stderr.\n",
      "W1105 08:00:28.866798 140736006943616 deprecation.py:323] From <ipython-input-1-aeda475fcce4>:10: read_data_sets (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use alternatives such as official/mnist/dataset.py from tensorflow/models.\n",
      "W1105 08:00:28.868058 140736006943616 deprecation.py:323] From /anaconda3/envs/fm1/lib/python3.7/site-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:260: maybe_download (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please write your own downloading logic.\n",
      "W1105 08:00:28.869129 140736006943616 deprecation.py:323] From /anaconda3/envs/fm1/lib/python3.7/site-packages/tensorflow/contrib/learn/python/learn/datasets/base.py:252: _internal_retry.<locals>.wrap.<locals>.wrapped_fn (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use urllib or similar directly.\n",
      "W1105 08:00:30.066245 140736006943616 deprecation.py:323] From /anaconda3/envs/fm1/lib/python3.7/site-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:262: extract_images (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use tf.data to implement this functionality.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Successfully downloaded train-images-idx3-ubyte.gz 9912422 bytes.\n",
      "Extracting /tmp/data/train-images-idx3-ubyte.gz\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "W1105 08:00:38.149729 140736006943616 deprecation.py:323] From /anaconda3/envs/fm1/lib/python3.7/site-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:267: extract_labels (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use tf.data to implement this functionality.\n",
      "W1105 08:00:38.153401 140736006943616 deprecation.py:323] From /anaconda3/envs/fm1/lib/python3.7/site-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:110: dense_to_one_hot (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use tf.one_hot on tensors.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Successfully downloaded train-labels-idx1-ubyte.gz 28881 bytes.\n",
      "Extracting /tmp/data/train-labels-idx1-ubyte.gz\n",
      "Successfully downloaded t10k-images-idx3-ubyte.gz 1648877 bytes.\n",
      "Extracting /tmp/data/t10k-images-idx3-ubyte.gz\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "W1105 08:00:46.350207 140736006943616 deprecation.py:323] From /anaconda3/envs/fm1/lib/python3.7/site-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:290: DataSet.__init__ (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use alternatives such as official/mnist/dataset.py from tensorflow/models.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Successfully downloaded t10k-labels-idx1-ubyte.gz 4542 bytes.\n",
      "Extracting /tmp/data/t10k-labels-idx1-ubyte.gz\n"
     ]
    }
   ],
   "source": [
    "#from __future__ import division, print_function, absolute_import\n",
    "\n",
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "# Import MINST data\n",
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "mnist = input_data.read_data_sets(\"/tmp/data/\", one_hot=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's give the parameters that are going to be used by our NN."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "learning_rate = 0.01\n",
    "training_epochs = 20\n",
    "batch_size = 256\n",
    "display_step = 1\n",
    "examples_to_show = 10\n",
    "\n",
    "# Network Parameters\n",
    "n_hidden_1 = 256 # 1st layer num features\n",
    "n_hidden_2 = 128 # 2nd layer num features\n",
    "n_input = 784 # MNIST data input (img shape: 28*28)\n",
    "\n",
    "# tf Graph input (only pictures)\n",
    "X = tf.placeholder(\"float\", [None, n_input])\n",
    "\n",
    "weights = {\n",
    "    'encoder_h1': tf.Variable(tf.random_normal([n_input, n_hidden_1])),\n",
    "    'encoder_h2': tf.Variable(tf.random_normal([n_hidden_1, n_hidden_2])),\n",
    "    'decoder_h1': tf.Variable(tf.random_normal([n_hidden_2, n_hidden_1])),\n",
    "    'decoder_h2': tf.Variable(tf.random_normal([n_hidden_1, n_input])),\n",
    "}\n",
    "biases = {\n",
    "    'encoder_b1': tf.Variable(tf.random_normal([n_hidden_1])),\n",
    "    'encoder_b2': tf.Variable(tf.random_normal([n_hidden_2])),\n",
    "    'decoder_b1': tf.Variable(tf.random_normal([n_hidden_1])),\n",
    "    'decoder_b2': tf.Variable(tf.random_normal([n_input])),\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we need to create our encoder. For this, we are going to use sigmoidal functions. Sigmoidal functions delivers great results with this type of network. This is due to having a good derivative that is well-suited to backpropagation. We can create our encoder using the sigmoidal function like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Building the encoder\n",
    "def encoder(x):\n",
    "    # Encoder first layer with sigmoid activation #1\n",
    "    layer_1 = tf.nn.sigmoid(tf.add(tf.matmul(x, weights['encoder_h1']), biases['encoder_b1']))\n",
    "    # Encoder second layer with sigmoid activation #2\n",
    "    layer_2 = tf.nn.sigmoid(tf.add(tf.matmul(layer_1, weights['encoder_h2']), biases['encoder_b2']))\n",
    "    return layer_2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And the decoder:\n",
    "\n",
    "You can see that the layer_1 in the encoder is the layer_2 in the decoder and vice-versa."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Building the decoder\n",
    "def decoder(x):\n",
    "    # Decoder first layer with sigmoid activation #1\n",
    "    layer_1 = tf.nn.sigmoid(tf.add(tf.matmul(x, weights['decoder_h1']),biases['decoder_b1']))\n",
    "    # Decoder second layer with sigmoid activation #2\n",
    "    layer_2 = tf.nn.sigmoid(tf.add(tf.matmul(layer_1, weights['decoder_h2']), biases['decoder_b2']))\n",
    "    return layer_2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's construct our model.\n",
    "In the variable <code>cost</code> we have the loss function and in the <code>optimizer</code> variable we have our gradient used for backpropagation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "W1105 08:00:53.102693 140736006943616 deprecation.py:323] From /anaconda3/envs/fm1/lib/python3.7/site-packages/tensorflow/python/ops/math_grad.py:1205: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use tf.where in 2.0, which has the same broadcast rule as np.where\n",
      "W1105 08:00:53.156349 140736006943616 deprecation.py:506] From /anaconda3/envs/fm1/lib/python3.7/site-packages/tensorflow/python/training/rmsprop.py:119: calling Ones.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Call initializer instance with the dtype argument instead of passing it to the constructor\n"
     ]
    }
   ],
   "source": [
    "# Construct model\n",
    "encoder_op = encoder(X)\n",
    "decoder_op = decoder(encoder_op)\n",
    "\n",
    "# Reconstructed Images\n",
    "y_pred = decoder_op\n",
    "# Targets (Labels) are the input data.\n",
    "y_true = X\n",
    "\n",
    "# Define loss and optimizer, minimize the squared error\n",
    "cost = tf.reduce_mean(tf.pow(y_true - y_pred, 2))\n",
    "optimizer = tf.train.RMSPropOptimizer(learning_rate).minimize(cost)\n",
    "\n",
    "# Initializing the variables\n",
    "init = tf.global_variables_initializer()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For training we will run for 20 epochs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0001 cost= 0.206291348\n",
      "Epoch: 0002 cost= 0.160441101\n",
      "Epoch: 0003 cost= 0.143924013\n",
      "Epoch: 0004 cost= 0.130115047\n",
      "Epoch: 0005 cost= 0.123792894\n",
      "Epoch: 0006 cost= 0.121629715\n",
      "Epoch: 0007 cost= 0.115214400\n",
      "Epoch: 0008 cost= 0.112786144\n",
      "Epoch: 0009 cost= 0.108993307\n",
      "Epoch: 0010 cost= 0.110740095\n",
      "Epoch: 0011 cost= 0.103984326\n",
      "Epoch: 0012 cost= 0.104051530\n",
      "Epoch: 0013 cost= 0.100393526\n",
      "Epoch: 0014 cost= 0.099269867\n",
      "Epoch: 0015 cost= 0.096566103\n",
      "Epoch: 0016 cost= 0.095864139\n",
      "Epoch: 0017 cost= 0.096248947\n",
      "Epoch: 0018 cost= 0.090926774\n",
      "Epoch: 0019 cost= 0.090809189\n",
      "Epoch: 0020 cost= 0.090478644\n",
      "Optimization Finished!\n"
     ]
    }
   ],
   "source": [
    "# Launch the graph\n",
    "# Using InteractiveSession (more convenient while using Notebooks)\n",
    "sess = tf.InteractiveSession()\n",
    "sess.run(init)\n",
    "\n",
    "total_batch = int(mnist.train.num_examples / batch_size)\n",
    "# Training cycle\n",
    "for epoch in range(training_epochs):\n",
    "    # Loop over all batches\n",
    "    for i in range(total_batch):\n",
    "        batch_xs, batch_ys = mnist.train.next_batch(batch_size)\n",
    "        # Run optimization op (backprop) and cost op (to get loss value)\n",
    "        _, c = sess.run([optimizer, cost], feed_dict={X: batch_xs})\n",
    "    # Display logs per epoch step\n",
    "    if epoch % display_step == 0:\n",
    "        print(\"Epoch:\", '%04d' % (epoch+1),\n",
    "              \"cost=\", \"{:.9f}\".format(c))\n",
    "\n",
    "print(\"Optimization Finished!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's apply encoder and decoder for our tests."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Applying encode and decode over test set\n",
    "encode_decode = sess.run(\n",
    "    y_pred, feed_dict={X: mnist.test.images[:examples_to_show]})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's simply visualize our graphs!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlIAAACNCAYAAAB8KJSgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nOydd3hUVfrHP2dm0gsk1BAgoRepooigYqfYXcu69nWXVSyrP3V11V1dV3ctq2tZy6Koq2LHDmJBUVR6l94CJHQISUifmfP74713ZpJMkkkySSbhfJ+Hh8mdc+8975z3tO9bjtJaY2BgYGBgYGBgUHc4mrsCBgYGBgYGBgYtFWYhZWBgYGBgYGBQT5iFlIGBgYGBgYFBPWEWUgYGBgYGBgYG9YRZSBkYGBgYGBgY1BNmIWVgYGBgYGBgUE80aCGllBqvlFqvlNqklLo7XJWKJBgZWz5au3xgZGwtaO0ytnb5wMh4REJrXa9/gBPYDPQEooEVwMD6Pi8S/xkZW/6/1i6fkbH562ZkNPIZGVuXjHX91xBGaiSwSWu9RWtdBrwDnNeA50UijIwtH61dPjAytha0dhlbu3xgZDwi4WrAvenAjoC/s4HjarohWsXoWBIa8MqmRSwJeCgnWaXqEgoBruIIl7Ely2ddKgBer+meliyj0VM/WruMLVk+65LpixgZIx0lFFKmS1VNZRqykAr24CrnzSilJgGTAGKJ5zh1WgNe2bTYo7M5wG4GqmNYoGdTThkcgTK2FvkAvtEf7OcIbEMwMkY6TF8UtBYZj2Q9hZYtYyAW6Nm1lmmIaS8b6Bbwd1dgZ+VCWuspWutjtNbHRBHTgNc1PWKIo4TiwEtHpIytTL5ojsA2BCNjpMP0RUErk/GI1FNo2TLWFQ1ZSC0C+iileiilooFfA5+Gp1qRgWRSKOYwxboQLQtuI2MLQ6B8Xu0FSKUVyQetvw3ByNgaYPpi68CRIGNdUW/TntbarZS6CfgS8eJ/RWu9Omw1iwA4lIN+ehjLmEsxRQDvNYWMWQ8dD4AnVtjSDkftY97Q6RXK9Pr2WpIWxgHQ6Zmf6/2u5pKxqRAon9XpD7Ym+aD1tyEYGVsDTF9sHTgSZKwrGpRHSms9U2vdV2vdS2v9cLgqFUlor9IYrcaTSBuMjC0Ttnxj1ASA3c1dn8ZAa29DMDK2Bpi+2DpwJMhYFzTE2dwgzMid0QeAX4b9p8p35ZVc+dad8jLTjkkD4L2vxwLgWbuxcSvYxFAjjgJgxqdvADD4xZsA6Pb3+jNwzQVn2zas/09PQNoO4L69I1h1eV8APGs2NFvdDAwMjiy4OncCoKxPlyrfRW3IAWD9n3vSdo3ElKWuLQHAMXdZE9WwZcEcEWNgYGBgYGBgUE8YRipCkDujDz8Neyfody8e6smT884AIDNjHwBfDfyQy5N2AfDwNe0B6HlX62Kk9h6bDIAbDwDxO6tE2LYYeHt0ZdXJ/wX87OJDHZcw9ILRAHRrgYyU55SjAbhpynsAvNCnd53uL7h0FG2X75dnrd8U3so1MQ5dJX6NCx55AYCBz00GoPujC9Fud7PVqzq4MiTguuO7hwD4fslAAPo/fwjP6vX1eqazQwcADkzoTcq7SwHQpaUNrapBmJB3xSgOTBRm6e7hswC4KnlmlXJT87oDcGHSR6RcHFvhu7PTRzRyLVsmzEKqmeE+TRTz26HPAVEAPJUr5p7vLpV8K+zcS9/cxQA4YkWx/7FgMPe0XyXPSIm8gTocyB0iC6hstwzG7abOa87q1Auubl0B6DGlZS8UgmHbOAlpTnUertf9u88qo/xKIcVTzw5btZocrvQu/P2vL1e4tubG5wGY8MyJ6IKC5qhWtXB17sSDcyR4pV+UF4BTD3QGwLO67psxewF1+Y+yeBoV+xE3rvqDfLkscnyQne3bAbD+3905uY/ImTO2HGh9Cz7H0AGsu1kSYM498ykAOjgX4QjBCHVdm+3Wp9gayxn4YUx7BgYGBgYGBgb1REQyUgd+LzR59ytlF79ubyfKSoWtSX9b/o/Pll2wd/maZqhh+HA4PRoABw4fEzXn3MEAeLZUpdg3/W04AG+lPgFWkrOus1rfeliPGcbcs58EYOwPNwPQm5bj6Lj9r2KyGzFe9POxtLlByyWOFlPtjr9I+fYrhV2M+2RhY1ex3lBRorOnnrq8Qc9JWhbLJdd9D8B3bYW58xzKa1jlmgF7x2VwZnx5hWtHL74UgA6HI8dk6+qaDkCbd4sYEu0EoN831wPQ5+ql9X7u2ocyAbgkUcxFRz/1J7osi5yAkL03Sd+6/49yGs1Z8V/5vju//TkAuHOq5JNs0SjskcSGCS9Yf8WFdM+LhyQYZtq2Y6st04bIYtYdw8QkXdJZ2Les8xUXjVwEQLkWHf/ujZEApH2fh24khrT1zcAGBgYGBgYGBk2EiGSk/nTnWwD8KiFXLvQK+PJk+S/LXQTA0/tOqfPzF+7NACDhiTYAuGYvqVc9w4G2r4vfz0WLr0Dl5gPg3pVVbfnfTfwGgERH6065f3BgHGnOeADSP4hq5trUHSv/8CwA5dpTY7k5Q6fJh6Hy30eFktLilYLzcX3bfHpZEwouECfzZ9JFxgEfS1qKPiyo03NKUzS3pKwDYE7SALnYghgpR7zo57hbfqzyXcw7KfJBR06ARO4YcTD/OPM537UB9+0FoL5elvr4oWw6W4Ioxq66GIBur6yjZq1vGjj7ysTx8u3iIzQsWqY7b0CZXS8kAZD2h864d7WstFaurumsvUuY3E4/S5qC5LfnA+Ao1WwoLwNgh7stAN1ch7jml6sByF0r/mKdFol+tv15B/qwWHnaHIos1qky9JhhAGy5Ed46/iUARlgMa1DcKex+8R1lTDkkDNbzKyRlUJ/r1uItKWlwnSJyIfXMPb8G4K9DhDBLWavJHSCKEj1EokweG/QhAP9OW8CMokQAzoqv6vRarEWZFpQK9XdybDmkyYDf+1JxiOxb+5mEjY7a8ghlPSzmzuva/su6Esvtu0YBkPTNWnlGo9Wu6XHa5Hl8XCgDQOIcMXG2BPmi5shCKErV0LEtLCvzklUujroXJBwE4JJEmdgueWNKREbI6DHDeO7RpwF4M182JP3vE92ta/scf+Yv4axak6N0tCz+Huo41XetyCvjTfJb85ulTsFgR+jtO88/YRzzLzGXd95RPxOcPl5W/vdN+5/v2uEZ4rCecGBLvZ4Zbqy9WxazQ2qYZBeMkE37hnllXPjG/wHQ82FxIQjHBNsYcLYVAmDkjK183F5OZhmz+KYKZWK+WMSdZ10D4IvCdA7oQ+r6zQCkeivON5EcruQ9QRZOWRIIy4wxshHo5YpDDlWBr4vFfHnPmvM5tF3mjV/Ol43eX/bIPPlY58UMjdsGwJMj3wXgz7ddQ9d/NtwMbUx7BgYGBgYGBgb1REQyUgkfLLD+919LrlTm2c4nA/DQmEySvxcq8rGTq+axcRULkZuwUnIutfthOoOjLYf1rJZhMjp05fH8dJUwUW0cEpI6r9TJ8ofE8TwuP3Idk+sK51H9APhHx7eZmt+yHJCLzx/JtWnvA36TXjDT3qDZ4uDbYXYMMXny/Z9Plj3Nqouf8ZXL/rM4yYZjxxQu5P65iK4u2b/+381nARCVWzcTpCtNmItXu8+iXLfcvdzWC6syHRdtPN/6FDnOyzueFsZ+48jXALhv7zDSXxWn2/qyvDknC8M/JsbLoJ/FXNT92cjRU+fAvnxz2lPWX8JWPHpAGMTFh7rzbq9ZFcr3jYrmpcvFOfvRV84DwLt1W9NUNkTYqW9KPxBG6p7239LvQ6Fp+n9UtT0r5wNriSdfbHlrGNOqmO+kPS/begaL1vUAoP8fxSrToXA9HaxS1484HYC9twhzftsLTu7rNAeAucViOVh+07Oc/6a0t3tHdr3r2XJHMQMDAwMDAwODZkZEMlKhwL17DwAJ0/f4VuEJHxyotvye34mP0VHRLv51UFiPzFfFlh/J9mGA/UdrHxNl4+o5v6Pvx62HibKRc0Y73+clBRnWp+LmqUyIsFm0h56cwjHRZfbVCmU+Kkzjvu9+BcCAP4mDtSc/3/d9v42S+mLhudLOI2NK+OKGxwA4M/ZPAGT+Y0mzJQ60U5K8P/hxXs8bAkDUN/Vzhl/zoPjslGsPV2fJrtGzd18Yatm0OOvYFb7PeV7R0fIH5AwzRwQxUlqLf6nNji44kImzeG+dnuFIEqfs9Q+Ls+7H50pqEi9RdL94VbiqGjbsH9mOTJcEA0zacRIA2aPEh9aRUMSI68VH7I7fS1b+y5P2cpI1xH42XRJSrjlLmNNIcEJ3pqSw7u8yRqwfIMlel5RC/wdlDgscS1oyHAnCdG58UFIArR37HA5rLF1UKo7xl39yIwD9/raWvockUbW38oOAwUlyZuDXLmGtFj8+gnZPirXr/IRDVikVnnqH5SkGBgYGBgYGBkcgWiwjFSrsiJX/3PMfQKKp3n9adsHtdkX2kSNlXwsjM6//E9jp+ofOE3+EAbdvbhFRbHVF/kB/YsPl/5FojbZEdjt5rbBqPxvlx2+3jQeg4NI4+mYLgxis3eyozcmvif/U4j88RZpTfAGWXie+Hr/68Gr0irVhrXuocJwvZ+J1ccUw9S2RqSt184mxmbs3T5Nw+VJdzvYnZZedUFq31AnNidKJkrDwP+kv+a5lW7S24/vITxo7s//HXDdH0sZsLxBfkbKpnastv/tEzcTjJPnqp12et66Kf+mY5b8mhcjzvfHEgBdhMFb+V9iNVGsc8RYWkvaE6O5750hbXpb0OWjhNfaUCvumSyLn2JidVwxg/QUShfZpoUQjTj37DDz7NjdntcKOQ1Yy6m8vFp9gB/HMLpZUP49Mlrmv91cSERtsHFUuF45+VtqLj1MBePx1iSwdHL0XEJbSqYRDGrzgN6Tvbfhv2OoXUutuk2y+x8YIhbe6rJjUNUXNWaVa4eqZCcDfe4vjcoojliVWn874u6iPJze3OarWaCidIAPaJ2fKYPHg/hGkTl8JBKdtIx337JFzEvN/J6ZKT3Zok03mdFmw/OX8UTzSeVHjVK4OsM9Ru6/vDN+1rv+on1PxuskSlnxMjOjwc7kDSZjechZQNvYcWzVI5ZzPbwXqnkurKdDxWVmQfzdFNmOnxJUwtft3ADgs04b3yerzXTlQvkWJjbcLxITZ7h5XRPbPpF/t8n3OG1cIQOqrVcv9NeNT65PfODN3WX8A+uZGjutEwXF+94ant54GQNyG1rWIArCSkVOi/Sa3Aq/o7+7j5ESF4gslU3nvPgFtXCK6fXHGUm5s+wYAi8uk/JgYW0PjfeV/KpFr6Q+psLhLGNOegYGBgYGBgUE90WoZqdKzhOFYetG/rStCD97wxz8S93Pk7DSCodd74iQ3PNq/zr3MCpnvu6L5WYrGQPapoopDomVncXXWYDoWrmvOKtUZgUk4Vx5t7+DraPZQshNzObxVknru/Bt0Pj/YTY0HFS/tMS5eUlCMXHQVnamfebF95sEKf0/begztiZyz6EJF9PCKbPDasiL6PyNMYiSa2+0M+U+fcCoAfx+dSfaZop+bznkRgIWlondXfHV9lfv7vF7KjPdfqXDtsTXjAEhf0ThnlzUUBdPT4Cj5fM1AYQl/OFaYjH3DE9Fniy4OipK5YG15OUdZZ0h+NEFY8btG/V4eMH9lU1W7Wrw9Zgo27/HBwDcBOP7J2+nxqbgTOOfU/6zESELKJ6JPk666HIA3+7/JuQmiq7+6QczKHu3nQEu12NRjVOBSRj77mSiBGw8nr5Rk36k3Sk/VW8Kjv4aRMjAwMDAwMDCoJ1otI7V9gqwRE5UwUZdtPQOA+FkriJzTryoi92oJMf9bpyesK1L3q7NOZ8CfJOloJO54w4EOgyQc295tuD5Jac7q1AnrbxDbe23n6oWCrAvFp+qDDgt9p5fbz+1yf9P7i3kPSpjw3/fJ+Xq/6bWYH9LEmTPUsHA74OOnYe9YV6RvFs9vDy2MkSo5eySLj33B+kvaZ315RzwtwF/FThkT/+Ee+soJW0y8/ugKZfpSla13DOnv86V6aP8gADL+KAxlpKaO6fzpVjb8WdiaO9utAeCuj4VJDfT3unSzJJUtvqUDF7w9B4Brk3cAsPkW0dNeEXDaz8iYKN84kGKlwll36XOUXyLX7CS/bRbJd4e7apKtk3raryz0PWf/EEkv0GmONd5GmN56CwoAiDlT/p/U6ULWPpAJwJkjJM3GhryOAGzLaY8zWuQ/t5+who91Xlztswd+N4l+t4u1x72nbuk/akOrXEg5kpK48kQ5SDTfK+cl7f1HTwBiSiPTNOZK78KJtwgFXflA4nlretM3NzLrHQ64emTwr37iWP9Snky6qa9EdqReIO478bN63+vqJtnbC0Z0AeDFa5+vUmZhqQyOqqzppy17YPsqRxxw5w57i12fS2bluf89vtr7Dg2UySoxM49RXbLkWZWWgSpSdzQ1oLi9s4rJ9U9LLqQHzW/+aSxsv9/pW3x89bDkZErcEQGrixrg3rWbSXdKAMCr/5KcV32jZBGB9tL7KzHb9b9J3Ae8hWt45NtzALjufCvD+TGy2nx56Fl4myla1kaPz37PhrNfrHLd1sX1p1sRpKeH9ryFd8vC+NY1lqnr7Mjc0Hj27KXvDbLoybKuRSMZ5/vgzzz/1UeS3yxwIZXllqCy85+VPHx9nlqIx904Y6gx7RkYGBgYGBgY1BOtkpHa+MBRfN5edvbnbZRs0jEzI5vRWXtPNz7uXJHZOGXVxQAM+NOmVmvSA9j4hy6Mski43y+V/Dbd+KUZa9R0WPM3yd+z+sz/VPlu+uH2ALxwh+hB7NrmC5JI+ZuwYmMfuIyPBr0GwKP3V88aLi6VnbIHR0B+rYpZhLs/uyoiQ+drQun5h3yf15bJjrfryy3jzM66Yv8kYRxXjnqOLLeE38ftq5orLVKR+L4w/NfyfwAcvETaqyQvhgF3iknLU+g3e/W7W0yAp/W5EICvj5oOwP33O0i/sGnqXB363biMce9PAuCq/8g8Ee8o5ex4ORGgMktaG0bGCMP44/BpABz1+C30urPlWAFsbP2H6OjSY+2gsmjfdxc9JkxUl+ckXUtjEuCGkTIwMDAwMDAwqCdaFSOVd8UoAFZe+gyb3ZIh+/Cj4oMSw65q74sELDn339jO5TbaTJb9uruVJd+sDG+3Et/n4kOxNZRsXYiak8Y/06ZX+/1rOaMBiP0sAtJ1LBRHzzYT4cqTbwHgUJ+Yaou3e8m/u835UOLQlxz3WoUytv9VS4CzrzjYLz72TWwn8y8Oi+N1fc8cjHQUnXHY9/mi5b8DoON3LS/M3mamEt/3XwvG8Nv6mP+RtKudPuHRIdN5Pu1koPnO3dNut0/P3u7fxXf9mYvEx8kTJWzv6DtkrAg1ma/D4lK6Do3s+TEYdt45mi8vl/NI45Q/2ebTub0B6PyqZONvCta71oWUUqob8DrQGanTFK3100qpVOBdIBPxA7tEa90iZ/wSXcRqFlFKCQpFOj3orvpQrstYxXyKKaKMEpRSKS1RxlDkiyMe3eIMLX6EKiOVTxNuQWjtegqtX0bTF01fbCk4EmQMF0JhpNzA7VrrpUqpJGCJUupr4Bpgttb6EaXU3cDdwF2NV9Xq4UqXFfqtf3kXkORcv15xJQAdvqh9Za5Q9GEIySoFty5nIbNJ1Z3YRRapdCRT9ecHPQMP7iaVsbyTREdFlaUH/d6zT5IA2inuVYwwBM4O7f1lOrTFUV5A7KWZtOmRjre4lC33Pcfh207m8PdLcSQM56Sv0tmct5DtzXRm1vPHven7nP5F/cbWUNowS6/jIHurP1SsnnAqmfQC/RTyfzOqQpm/PTiVU+JKKlyLUs6AlAlV5dan5lT4O1L01E7+125OaOWLs+TsMo6reF2PGYb6aXmFa5EiY2XsOUVCrgPb+D/fSUqVuhwLE6qeNldfDMR/R8hRG7s8RbR7Kr6W0n40Z18MBzr8V1id4yb8BoAFI97ij3dkAtDrdmGkIkVPEz6oqHufDRWfoUeuXESRFn+2ET/cAEDGy0723yJ+YsKs1oxIkbEyys+U47c+vukxursq6uV2dxGf3iVH6MQUNZ1fdK0LKa31LhC7mNa6QCm1FkgHzgNOtor9D5hDMyyklMvF0M+zAbg48QAA0wo60ukvQlmGsq+LUXHEIOf5uFQU8TqJUorZx05GMBaAKKIpo/h8mlDGGR+8UuP3o5ddBsD+PckApHQQanrBiLdqvO+i94q5vvu73LbuAF9N78xpI88k/c49bGZNGGodOkrOkUzDJ8QupKFW5lDaMI0MNvFL2BNUPfLuRQBcYh0uDPDD488BFXNLlQfxdqwu99Sg2dfTh4pmlEjV01ph+Zg7KrlkVl5EQeTKWJLqd5RfUioT1IBHZdypS0B1qHra1H0xENl/FpPymBjRv/ml8TjrYNJrzr4YFnilT7Z7Qibp/W8Us/bX0p/PeesqAGKWrI5IPe3+pXVu3JUQr8Txeu3YqXIp4wxmZn5plazYF7fvTqWPL8GAIFL7YtbZspnJDFhE7fLIAvGqW28nfkbTn3dZJ2dzpVQmMBxYAHSyFln2YqtjuCvXHCjWhRRwiDakUkYpMUoUyZoEWryMWTvKWf5LKSOPjmHvfg9pnWQBE6Pi0BGbqrRuqK4Nrf9bhV9ga9dTaP0y1qSnpi+2HLR2PYUjQ8aGIGRFVkolAtOBW7XW+Uqp2m6x75sETAKIJXR6OGQM7cffO75R4dJz/7iYtivqHsrp1m5WMo9+DMOlokKOlwyHjOetuZzZgz6o0z0/D3+72u9sWrc84FyicQt+w4a7ppF88hWMnj6EgrJ7GfTaTaT/WPOeurHacPu58gPHKBcP7h8MQOIn4lBZ32mkOdqw57tiYl14RSwjY0pqKV0RdrLNKbtlh5c7Wawd/bdWn/KiOfW0XrDqVzkhZ02INBk7BphZP80fDvhN6/VBpMkXiMsvmw34M4Bft/gaMpBgA2e7VCnUUTLwe9ZWb4KMZBlDgeP7ZQCc/L87WfNbYaQKHpY0EMkXJ+EtKIg4GaMWS3uMWnoZ84+uOD+8kfk1NndSqiUY62wrIWf/WzZH/Hhj696yC23m3x/scvKPNwHQ66OmZ6MgREZKKRWFLKKmaa2twwXYo5RKs75PA4LmXNdaT9FaH6O1PiaK6qN8mhte7WUl8+hMdzoq8UmKJoZSLR3HmgRarIzl5ZotD39I6slHkThoCADOxCTc+fkAlOpiFMEXxy1BPqi9Da3/g64aW4uMLV1PofXLGIqemr7Y8mVs6XoKR4aM4UAoUXsKmAqs1Vo/GfDVp8DVwCPW/580Sg2rgXNgXwAmveN/7cBXbgQg8426HV+gtWYNi0kgiQzV13e9A13YxTYy6U85ZdCIMsaN28pR/5BVtQ7SKkn95bTyYP5PR829Vu7bnuC71vMDK3R54Sq01qxmEW1w0m9ZCiBsXblOwfXIdGJUf7LYhoumSSzoTBafrrvGzPRde+sLOXqip7t+SeFCacNdcqTAoWofUk941sjxCn/9v9+x4xxhXTZM+G9I905+Rc7I6vbwz9aV6oNfIkFP6wNvbEUmap+ntNqykSajHcBxXpcVvmsHyhKlrqXVy1EdQtXTpuqLocDrcbD3JvGbOut3cwH4eEsaQNBElc3ZFxsDvafs4I2LhSn+YbBYDcYNuZa1P74UMXpqw07h0PnmFM555VwA7smcAcDxMR5fkt97Z14KQO/bZK4MxkZFSl90pogr3a0LRPfs83MBHj0wAIA+vxcmrrliXZXWNfN0SqkTgLnAKvz1vAfxk3oP6A5sBy7WWh+s6VnJKlUfp05raJ0B2PgfCQHaeMELvmtnXiXnJ7lm1y2vyyG9n8XMIZE2vmu9GUQyqaxiPiUUU0YJHtztmlLGcCEU+WKJo5wyCvShGm224ZDPnpz6/iTqtCq3C3G/kgWEx2LI6opQZcxl33Kt9fCanhUOGfMvk6i9qGvkoNhZR73Lmb8Ije59TVwKtIKU5RIgUZOJxEZL1dO/bhFH5QQl5oTLXrsNgO5/+7lK2UiTUblkV7P9HTlrcM3oNxk0/3IA0i9cXefnRVpfDIaxK4VtsA/7daB8Zr6jfvit1PkByQjuWb+pyv2R1hfDAeeAPgB89o1Ehg94/QQ23/16xOhpTdhziyyCC44tpv99Yo52b9tR632R0hdzr5ZIxIX/lLneE+CuMvYWiUhMmN54Jr0Fejb5+mCNfTGUqL0fqXy2gx/Nr+FhQFvVntO5KOh3dnSC9WPWqCyRilDkA5GxpSJUGb/RH7TY03Zau55C65fR9MXW0RcTBnbndNV69RRaf18MJ1pc1IQdMj/7nCesK83njGhQP9gmkfWSDoRotrW6swST37bMy5a/5wWMJIEt1rdbfOVam9zB8OBWMTEUPi8+Ft2nV2WiIhXaOi0+825hYAb880rU8qTmrFKj48t7ZZJc82cx381b0J/+T+8EoNfu9QB4SuoWVNHSYTPGl245E4DPhr/MdaMmy5fzVzZXtUJCp2ekv3Wibqk6IgW/uuMboCITBdD7s+vp24hMVF1gztozMDAwMDAwMKgnWhwjtXOMJOMKzGg6rcDKOpwvIf+tIwOLgUErwWmSuDKB7GauSP3h2bQVgO4XN3NFmgD22Y77PpO/ezO/RTIZjYGiC2R2WfBzF3L7SXBPSt1imwzqiKFx2wFwKuF95pcIjz/wsb0Ro5eGkTIwMDAwMDAwqCdaHCNVGf88MJB54zIB0LtWNW9lDAwMDAxaLTz7Jcp2St+epFC/VC0GdcOt064DYN3vnwfgt6/cDEC3LZHja9niFlI97xblnXj30QFXdzdPZQwMDAwMDAwaDRn3y4Jp3P3DAOhG5CygbBjTnoGBgYGBgYFBPVFrQs6wvkypfUAhUP9DqpoO7alYzwytdYfablJKFQDrG61W4UWdZWzhbQitX8ZQ9fRIkNH0xciB6YvV4AiRsXX3xaZcSAEopRZrrY9p0pfWA/WtZ0uRD1q/jA2pp5ExctDa9RRav4xGTxvv3qZEa9dTqF9djWnPwMDAwMDAwKCeMAspAwMDAwMDA4N6ojkWUlOa4Z31QX3r2Zu8CnMAACAASURBVFLkg9YvY0PqaWSMHLR2PYXWL6PR08a7tynR2vUU6lHXJveRMjAwMDAwMDBoLTCmPQMDAwMDAwODesIspAwMDAwMDAwM6okmW0gppcYrpdYrpTYppe5uqvfWBqVUN6XUd0qptUqp1UqpP1rXH1BK5Silllv/JobwLCNjMyFcMkaqfND6ZTR6amSs9JxWLZ91j5GxmRBOGdFaN/o/wAlsBnoC0cAKYGBTvDuEuqUBR1ufk4ANwEDgAeAOI+ORI2Mky3ckyGj01Mh4pMhnZGw9MmqtG8ZI1WGlORLYpLXeorUuA94BzmvIu8MFrfUurfVS63MBsBZIt783MlZAS5WxX0uXD1q/jEZPjwgZjZ76YWRsRtQmY11Q74WUUsoJPAdMQFZxlymlBlZTPB3YEfB3NvWscGNCKZUJDAcWWJduAj4BVgKjMTK2RBkVcA2ggR+BK1q6fND6ZTwC9RRav4xGTyvCyBghCCajUmqlUuoVpVRKbfc3hJGqy0pTBbkWUXkXlFKJwHTgVq11PvACcDkwB6H8HsHI2BJlXAB8iyz2c4ASWrB80PplPEL19EiQ0ehppUcEuWZkbGJUI2MvYBiwC3ii1mdY9sH6vPwiYLzW+nfW31cCx2mtb6pUbhJwG9DFiTM5nuQan9t3SBEbVsbXei2cqO755ZThoZwhQxRZO8o5kOu9iiNQRku+SUCCE2f/liofwJKVpV7ghVbehtnAJ61cRtMXTV+s9n2RIOORrKfQsmUM/D5rRzn7D3qCLQh9aMhC6mJgXKWF1Eit9c1ByrqADUmk9DhOnVbrs7/cuRyAcV2G1alO9b2vOuzR2RxgNwPVMSzQsykg9yqOcBmTSCmvl3xKQQi61pjyAXyjPygEXm3lbTgfWNzKZWyevhigx80tY737YogwfbFCheT/WsawiNHTMNxXHY4EGQOxQM8mXx+scSHlasDzs4FuAX93BXYGK6i1diulbgJmhPLg8d3tg5fdgPxIofxAIf+IDqf87/XUWCyGOEooDrwUNhkr17VeMobYuWtCXWVMVqkhPbeKfDnLQpMvfbj9Nv/FBsgZRL4CmrsNu46wPtWsf6EiiIzfIXJWQaPKGLDICNqODUDE9MUAHQxaJry62jh9MdzjaYiI9L6oYmLk3aWlwSoUSjUiR09ruK+hOBJkrCsa4iO1COijlOqhlIoGfg18Wl1hrfXMBryrWZBMCsUcplgXomVCOOJlbGkIlM+rvQBtaP1teBatX0bTF1sYjtC+eMTraUuUsa6oNyMVsNL8EskV8YrWenU4KqXd7gp/h321WQMTFUgROpSDfnoYy5hLMUUA74VLxsqoq4yOpCS8BUI6qBFHybW8IgB09i68JSXV3tscMo7rMiy03Xqw7+xrNTGJ1ZhcAuWzOv3BZm9Du/6B5s46MhkRqaeBda9JjhAZ4UaVsYbfO2RdDYZQyoegq43eF5sILaYvOpzBmagQEJF9sTbYfVAWtLXqbYuUsY5oiHmwQXmktNYztdZ9tda9tNYPN+RZkYr2Ko3RajyJtMHI2DJhyzdGTQDY3dz1aQy09jYEI2NrgOmLrQNHgox1QUN8pFolmtvWGgybpw2n1+XLAPDOttzSTtvBzjtGA+CwNlUdlomNP+ZwEcqdIBedsvPw7Dvg2301m4wh7NYdsbEAeEtKcHboAIC7r6QdicraK3/v3IUjMVHKWaxcUN+VMPiQNRgqwEfRqoezX28APOs3UTZO/AFj9wibyGZJueItLMIRK+3pLSryP8v2QYogPQ1ss5BQHaMItfsghQu16UQouhov0T7eoiKcvXsAkD+0IwBtFmQD4N4ZsFYIsvuPKF2161FPljQYIklPfQgmYy3saIV7AZSj+cfTIKjRzwt8TJQjTvqsssYYFRuLZ/8BAJwdrXF3R7bvtmaVMUQGu6FoiIzm0GIDAwMDAwMDg3qiVTFSDQqBbKYdoWOYJIT1Ll9D3hWjAGj7W2El2sZIZESvMcvYMFWYi60DXgbg3O/H80b3fwNw7cqrAcg7JAlYO+QkwMFDAOi8fHmRvRuGyNn9BsEXW+YDMKH3aHZMkZ3RE4OnAfDAvdcB0HZ2Kd5DeXKDvVuBqjuWSJBPWXsVr4esh48HYP21LwBw3fYTuCftKQDezpP2/fFaiSJy7TyAZ99+6xmR215QiYmy6vpljjCo49KH42zfXr6zdsm2D6QjOcnPmO4RtlHFxflZxkiSO8iu+ItNPwMwceBY2r8hu/mnurwJwEVP3wlAt3e1TzbtCXhWhOiqIylJXl9WJv+XlvqufbF+LiCRbTaDYTMd2iv1VVEuHJkWS+6VMcablY0uL2saAeqCgDZ0tm0DwMw13wMwPmOkr5h2l8sHq+8qpxNneucKj/Ls2oMuDfB1hIjQ00AmSrlkep+1fTEAE3qOYsu9Ek074KQtAKxaJkzqpNNmM+9gTwBWbu0EQGLbJNLOX9s0Fa8JITBRQed+VTFjgSMmBmXpNm3l//LObXDMXdbgKraIhVRtIZA1LqACqViovlGashMohbImEFXmT/Fw4o3HAbD/re4AlOTId/ED27N1giyghj0yGYBOz/zMnNX9ALit3zcA/DvhEnm+w4HXWkDZA17IjsDhQqWJJ+Q2tMPmdRFndl8HQP/oXAD2D5O2bLu0DQ5LBk+uLKhUlMs/sDUTAmW0BzFn964ArPlTR7aeKwuoxw/2AiB71GHit8u95yaL/IufzACg8K9dcOVai2FPEOf0ZsKXO5f72ki5ouT/aOt/p5OZ634AYOJRp8gNOhdtmSaVNRllny8DdVmypuNSmXyTfhJd13n5TUbl21BR0fJua/IPlNH3ewfUxaerdui8N49e8XJvB4foaNdzsuS79x3+scdOdxG4qQlaIdU4uZ8DFrnjz7kcgOJOYp6M2y6LV+ehAmYslEh13+LCW4Yuk8WFiouTa/0sPe2WQFmSyNdukSz8ne1TfZuAyoFDTYXANnRY5i4VLe1ceFJ/vp8yBYCxf5gEQGz5Qt8C0qFk0Wgv8gG8B2QMUk6R1REXi92K9XVSbygCxxvbzOxIthbGHi8zV3wNwISJvwHAW7KG9qtEsUaelwXAMxdOB6BEK46L3wzAF8lDAPj48+Nx9pWxyrNRFl5NPf6EOm/4F8JlOPvIgnDt7bKB69T9IAB7N7cjYYe0acwBkcMbDe291m+4WBaN9qaiLrIa056BgYGBgYGBQT0R0YyUK70LAGcd3w1H7D4APzVn7QJUUiIgq1LbObnghJ4kz98GgKerXNs1Wu5LyvaQvEKods+W7VVf2hS7YK1x9BDWqTRN6nXaldfhcsm7O87ZULFKvfxnPBaOkt39wTP70i1qFgBJDjEBFmTICrpLUUkAE2Xtm5qZzQjV3GqzHNvvPoZpHR8HINst11xFsqPWO3b6zEk286PLynw7bpvta+rdcKCMpafLbnjkPxcB8EK7NwFxkL8ieQUAo7fEs88r9e/gkLo+lvkhAMtf6sILN18MQOyuwwB4f9kI9j64NoY13LBYovE9jsPZTuQoGik7v6h8YSv2HBcPCCN16ExhS1N+2gEWo1beQXT9cF9rx1fu4HCayJ9stSNxcTgcIpvNxDX2jt/H+FmQdqzUVyqZCQKx44NB3N5OGGOHNaS2j5U223dI+81cQZg2mwXxmTMh/P3Ueq8zRcxZA5+bjPdc61X9pZ5p/5N6xFvME/h/F1dGN7x7rPE3TZzp1/1emClnUineA8L45GfKWNt5YVtivhMGx277Rh977L5vsU7juo7AmSQBN7qnsMJbz2sLwAtXvei7Ledki01LOZ7UpRbrVFAoX1q/hS4s8rEUjvQ0ubb/oI/p8lpjbZOZMy1Zx3UdgTNZZDw0UVxEyuPkuwMj/Kznll9Ju49+MYkF2dLex8QLw7TDLUxWN1cRUUrGoJxi+Z1SRuyjZK64i8S5hYF0b8lqHJmqwbiuI3x0jzPRas/usi7Yc2IK9ty/6TVJAdQ+pYC7+8i82C9K5vloJb9Fm8GKbdZcUuAV1vGr/EG8vUBcavp4+wOg5q2ocz0NI2VgYGBgYGBgUE80HyNV2Sk1YDfvShNfCneOZJ3f/sBoup8oq/6EKNmdFp4kPkDH/pDL5BxZUf5p/mwA1pWu4fq2OQA8tF9WmZe1EYe73Z54blt7KQDxzx0t/y/KonSIMESz35hapT4NRVAfLmvX7Zq9RP5MSPDt3rzW7s3eiZd07kO/qTcA8NJv/gtAF1cBfaNkhX7nbmE/nLZpt7hC+n4fKjsf2kfxNJS5CSpfbX4gFSqmfP4Mzo6yG7ri4tm0d4p8U3KF3cj4RGzdgc7NDstp1LP/AM727QCYuXJ21fo0ECEFMjicOPuI8+ZNz7wrde8rrE1qtovtbtkNflgwCICfc3vxXk+p64uHhHW8KnkrAJ3id/P1g2Kz33mR7ApVSht0iejEFxt/qr0+dUSNDpsWi+KIT2brZGmPU85ZCsDmY6U9ym4ZzPF3XA9Awl7Lebm4mJmrvgVg4kkXyDMOi49UQmYeBUXSfuUZwma41mT5nJZnbV3QNDKGwuoFSVngiBdGontqLokO2eHmeaXvrXhX2rjz4Xm++5zt5EgXz759VZydwyVjMPlshtZzQPpPcUY51x8/B4ANhdIW2TMt1tPlCjjeR9rBvWMnX2bLOHXCLX8AoG1n8U1MS85nk0N8UTLvEd3dMXkw3X4SZqjJ2tBiaO0x0xEbS/kQ6XubrpVxr+91EhyQ8dt8btkpPnxJmy1GaskBZn4tfdbn32fBW1rqY6TcW8XSoZxO37jZGOe81fhMSxcdcTFsvkuYmFsu+ByATwfKGFh41iBO+e3vAegUI/q9YeVA1jwt88dJN4hvWG4/+W0Gn7eW/HLR4bM7rgQgp7ANWy+U32fruR9XX596Iqiu2hYG67d1tkul6NhMAEb8XXTwlxHiO3v3h1v5qUR09IPRwjKuK+vM+Qmiy3/IlrP9/tRJfMWcKIZES1+4KUf8kU9ps46fP5HPX03/X71lbL6FVDX5cMrGH8vm06Rave6UHCyXX/gtE5Kkce/pIU5lL277EZCuPjnjBACOyRazVwfnJu7bK+WilChRrygxR6Q5y3hhoESBXXS2HFY9cJnTt6BpDARtmF17K/zpyxUEvt/GVqrStg4y/yIDcs4lMqm+d3AkX8+WAa/TIlGm9c+KM/OEf4wJupCpvGBq0AKqtpxGoVD5QSLuSgbbxzdu4/59MkgsvEqcH7+Y9XaV99m5Typ/DjeqyBgQMGD/jq5uXYiaIiYaewE1M2epdUM0v+8uC/5BS2Rw2vBeP05dJwuvbVeJ/Nef+iogE/KDaV8CcNKjoqd97831TYaNgcoyqqhoHBad7skVs8e+C/pz/+XSDq9aDscbn5OBKHG+k+S3ZLJypoieegsKfE7ZKko2Rpsv+QiAXt9eizde9LS4swziCUtKGtWUV9/zOH0DvMfj0+2y4wcAcGf3N8n1SP/9pDATgBV/el7e97S9KJEFlO+zHXUaZgTTU89xYvZx/CgT1+TR33KeNZ7ePHiMfDdUZNHrtoAdVWnlB1OJCYydJBNvzgXye2w99h0ALtlyGmMyZQG16MbBAHRaVFrRVBlmhNqGhelS/77XSSTwOztEN7e5o1h/jJjtUs60dp879wac8Sp19204AyL67LZvbLeBYDJWXmSojHSGjhU3EHsBVfpVJgAdn4ojepa4Fdh567xbtjP+I5ExMWkjAD+88B0gQUyHLbeRnfkynxYWx9D3+oXy8nPDJ5uNYDL6XFIsHB7dgzv+JZGwz/XpC8C9WyxXHrw82FPIkLKvZSwqm9KZN+ZJ/isdL+3fa47M/bmeIhzIxvCZdHFB+OhwR2K+WNRgWYxpz8DAwMDAwMCgnogYZ3N7tZ3f3eUzUdnU3+MHe/HH9b8GIPFb2a1esfYqAE5PW+8r12eOmBXiF8bTbo08ZNcooZjvv34NABd0HcnU7cJmKbflvOz2+B34wnxqfTA427bBWygmgJooXKdl4sztr5hvlRuz8kIAdq/tSJ/pQmE6siqetOAtLKy4gwbZSdkyhoOerYlxCnaWWNcRFc61AnCmihnPm1fALMt0MGG8mGJPT/qFGflSTx1dNzVtzCy8dqg8DgXeijKOXNaHtzOFrem1U3ZB5dbvsMdT7Ct35q8k71eXfbso7ywmnm7TxAmSU+W/S7oe79tBv3O8hGrfMPaPpO4Qk3VTZBpW0VEiJ34Zr9ueyI5yMVEl/CDmuLRnpXybORuZWZNJwuus8GfvK5ax6U3pb1rZKQjcPv1o6mzKtowTeo6qEszgsBm23FwfUzF2kjAVJ8UW4FTSfosLhGG8JtlinGthZptCxujNMj7MsOT7qiiKfR5xFt/69lAAet+6C5DxwhdSbjE0qlN7dpwhe+7TB1U8Ui3vhAMsf0Tyo8VZRGLsmhy0ZcZsVPkCgksqm9kmHnUK+86TNnzkn3Kt3BrTH9h+Ll/uFLZ3fI8Ah3HLPGi7C9io1ok8nONpCKjsmtH7rVHE7JF+E/+5MFL6Xfm/47aDvr44vrs1F3i1bwz2HDpU4dmdnvmZvP6izykdhfV2fpTqS38wrkvjyARUaEc7XcyMH8WUOKd4Jb2jxI3nySyxOLx2UE70+PjrUWzcKVaYiYNF32LycnBbc55tSrfx626jfRYCj9Uv39w1CkeCPN9kNjcwMDAwMDAwaAY0CyMVLMmWvaPo/MUOdJKEZI67V8rsu/54lLWxi94su4Pk3RKi+lOH45i4SpibnvuXV3lXj61Wcq7fiv13Zs5SNltMVLevLXt3UVHtp9XXMcq8pkRinvzDNZ7T5EsQmCi7xh6jtzPiAXE2bz9FfKX6pxX5kt55rBX4hJ7ig+OIBa8dxhx4plQYQ5BrTJSmdRBWwf8D2g7Zeof4zGh3eUACRHEkXFqcyRs/iq2+36oVlZ7VNAiqp/buNMBHyi6T924cXV2yw/VYPmpFWso/f2A0S38vvl5qsRVe2683jp9XARBvhaafsvo8AD7c8TbJlgNzoZb9Tnl89SH49UWN7ehwgLULtstcsGYrqS5hQqfulp1hzzWyu/Xm59fcRtZv0nvONQBM/uUHcrfJsxKzLKbP6Qx7GHltSf2q6mqJb5dspynROcLqaI/H5zMTWy7+I6XazZZy+Z02TRJ/lHHLbRkaP+VITfI5YmLQqaJbdpmp23+kSIt8cQusMzkD0h74E43KmOyNdqGT5fuleyUoYlCOJPR03tqGbl8JFRU9X4IjPB6PP6lhmBBUxiD+SnYZPbo7Z/X7BYBjoqUupZbM6/d2ZMLZJ0q5UvGHcsTGom3fVCvp5sSxwv4r1/bgPlFhTulQc190+upnl+kxO4eUGJnXsp/tA0CbmSKzt6wsYEwNUneLfZsw4TIAtj7SlsSt8vtsiZUUFwN+3OPT+3AhmIz2OOpok+xj6O0y03b8RLzFVv9SJik6Zk+Rea7vrBzG32sHTAX4jtoBMh2FnbMDQL7cuRyP1WQLS2VsLXiiG7HFSyreV492NYyUgYGBgYGBgUE90SyMVOCKtLKPkPdQHg5rR1B8nuz8On+3j5nffQDIGWzg9/1xri7z2TuDwlpltnVY50ChuTlDIlUSjpLIGU9gxFww1CPpYU0yKodC60rsgtZ++/7AsQAcGior6vi/J9E5y2JvLPu9e8++KpF5drJSFR2F10odYdvVIbyRJjXJBwT9zexyZx0rBnePFcqP1r56bntboofaOj+n/SLRA2+oUVxhPlqkRhmVw3eExJ7JoqfDu64jxvKVKfLKLvjNfIk0+eapMbRbK5FStg+GNzAhrKWn2zYIu3qwr5dfd5OIlGe3SaqDzj8cDDu/UaOM5eW4ewmjW/Ca6Nbo+PkMiZbd3KOzxQ+MHPE/rC3azj7GYlC6+OP0i91JylnJ8p0dWVSXtBkhos66qpQvLctZY4SR8tgpRQLGmmOX+++7Zvk1AKSvWl+lXFCE8Xy2muTTHi86Xnb0cd9LqoMCr4MB0dIWrmKb6he91V7tG1ecvYU53j80GZd17mfuAdGDxFXCvKZ9n4deJn5T2mbS3eVhZ2tqbUMLNpu/b0g8T6WJL1G5ln663+qT7d6N96WasZM40zYJe0R2pwhL59prHT8VF4dugvMfa5RRe33MzYbXZVy4Pe1rbmwr57JOXGqNKZae1jbWO2Kl/RwHRa7yNkn0uFsY1l23j7YKOXzPCxdqnBejosg5WX77796VMa+NI5YoJXK/slssFKnrZZzx5Oz2+wBbY79yKBw9JYLvmo/FDy5R+a0EE9Pltzv4uYzLHeauxxtl+ZA1IFq42Z3Nq9B8SqHzpHET11l0/4Fcf84jrwhr5zWplgq1FH7Tb4WmtDNIX9N9BJv/ZVGDj/vPD2qMXCA2qpiHAkKo/SkAvL5yrkzpFD8/KbkxJg4+FY99dp7dQZTyZfF1dpA8LsUDJetu1FeL0aPFiVQtEVMZAU6kDZJRyeIsGJ1esVzVAWdi/5OkKof3ABXb0GllDO5+sZi6YjeWk7r6sPWI0AatWnWiAaiipw6FsmRM2CWd+cIOSzjsFQfXb4qlTeyB7vP3V/jMHd5CK3Oyw+mbtHSRDFixu0QfJmecwLQdMpisLxcTrzcxmi8tR9PGlFHZZ5NldKW4kyya/t3vFQD2eRIY8JOc3dXjW2nH2nIh+RbJN0pupctSJK/UM737s/ktuaf3v60B0ekMr4yVTNo16qoNrZk4RHLQeA5KKHXgIczekZJOYNEw6U87sxSpr8iissYJLKBPBMufFw5Ufp6zfSqlCbJIOr+TvPPbon7csEHMdw7LAjdz2VeA9FGdaW10VsrY4XzWwQkp4jy/q0gWvrnfWibPZat9ctmH/Tri4nwHOjdFXwyEI0H6SkEmzCmWjVjvKBlHurukjWJy3XgHiRM1C2W82fBwJv16y+ZzyyJZcHZaKIvNhOnbfX1Cl9vt6220Ngz2TGe7VLzdZJP14YniYL25vANnXHINAFF7ZC6zHdHHpQ8PvtizFxwJVpZwK0VA3xsWknu1BA10+1ACWrx79zepjLpdW4p6ikLGWKZHL15mFUk77CiQ4KR502QsmnDmr/EmWWOVlY18/fMj+b8TZQF1XsJ+6xnyrLPTR/gCZJzXSTt6DuWFZV40pj0DAwMDAwMDg3qi2Rmpyqc3a7fbn+F7/SbAn9xPLlbvpB0I+5y+UWOFdrbNEaf/UoDjN+Ic69m7z1cHv8mm+ozr9UVQs5C2duCWPM62bXwnrLuzxOQzoZ/lEFmc76MwA011jjjLGXmoyDrnpZcA6PHxJDI+l91IvFVm5prvwyOjrrrrDp7ZvOJuSMXE4C0uqSCzjwFxudh+sSTi/PhmSR9wxve30G+VZTIKVl+bybMYnS9zljVpG2qvRkXJTj/xfcne/PGtR5PcSXbiL/QRU9WUTsKIeov9yRj9zITXf2aedW3NZEnkOOuaGJyWsWFGnrzzyw9f973fDu21s4aHVUbLSbUkoy27R0n97u8pDMaoFeX8beinAEzdKKYfXyJDFWAis+RSTieOTAlpPu8SSTtyX3thOp78pScvvyO7TbVa3v3F1gVVZGnQjrGSHoaqq75rlXTVkRDPxsnSLovekbb+X/5A4r8Ws603mO7Z7W39Jl/mLPG/39LjcLGplVMXeDulUtxB9PTt/jJO7L5tNPn9pA/3fVWCV4bHTwbg8L0adxuROamzmNn/2ecjkhzSd7/Kl6SbDz0yA4AJ743ymd7tczK/2PRzeNuwGhmDPctOdNrzw8PkXCDzxj97SZDHhhdkjulbUEx5srD5Wa9ZjvVlcLBYdHHSWcLO3XnlZgBO8kwicbWwG3aAzBdb5jfJeDOhj7ih6C4dWH+LjOV39ZAEuN0XJLB/qNS5449isbADjpxJ0ag2wh7aCVaJiwWLUSvvKm4jdjbvkffcQIc5wkRRKqzQFxt/ajQ9DZTR96z9uahoYcou6Srs2JNZ89jrFnNy8gRpjxNmSRDA7z74li4uSRTczSXtnuT4kVSHZcmyxk/bNFj2dQbFT1qnKuT84qtDOHTVMFIGBgYGBgYGBvVEszNS/lWffVCct4IfEFRKHlaDQ7EveV5SEodeFtvpW93ESX2XW8q/8co4Vs56vsK7KzjANYJNOGi4p+U4qCxHN29hMc7Owl7ogeLg68iRXRCJCXgPi1+NipHVtkpOYtvFssPsPj4LgHM3jgcg+qCTOS9bx8VYOxRh/MoaRcYqzwnw/fGFKAc68gXxn4o9eX+FRyT8Euv/jaydbmCy1Mo7o6ZqQ9uZVUW5fD5Pe24R58zN24pYOF/OoetlHbfgWbDKqnIQf4UA3xzPUGGw7t8nun58wiZSnLLbfOdn2Z0t++9AQNiccDJRNvzP8tdVO+Vzn0XSn9q7cvk6V3ydcq+RnX6HWZavYWwM3mSpc94A8WfIHeDgtks/BuA3SVIuz0pk+uPBXj4GbvwjwqBI6L301cbwWwzFn0+XluKx9dVRMYmoo0M7/jvqdfls3ffBjqNJbiN+OOqgtF8FXQ3wr6pch3D79fmfYx0jsnEbbUplnNDDxLcr/bNdpL9l+ekNsMLmtwobXpoahbuj/P5pSeKr6kT7Ugjsi5ez5o67S9KxpLgX+eWz0iaE6hjecBkFyuXyHS1is/Rqy06m3ixnO+56QPpshwVSxp0YjatYZEz4RfS1sLuHP/ScC8BBj7Ais4ul7XdM0Gx9/kPAzxCN73EcYPnrNuqcYY37UU4G9xTGqOsikXHW6v7QV8bZJCswq6i91LnkrHyGdZbyB0tFxsSoAq7uLH6Xz24XH8B+r0o79v4umxnzPqsio+Wn3ShnJlZ+lrt3FxJXiGxb35Gx5Zw5I4hfZyVNvVfKpSApGb45OJBTUmQ83OkW9vHU+C3EqIrLmpVlwqYWv5bG/OfF79hmbKVPVkoeXQ8Zm30h5YM1KM3avjgg9kDcCwAAG6RJREFUUiFg8rGd5KzJ1Z5ktcfjX1RZ1Pn6ZzJZOkgGaDsHxXel4sDdddomxj1Z9YdqTGfzypi1baE/a7AVLeMtKmLnueLA6Ra9JzpPKM3OX+/Eu0uUx+kShSnrlkJijnSiLfuEpnU45PdK3hxoRvTT7hWyjAN1To4VIr7MDjBdBItyqWxKcblIPVvOjHp1uSwakrZ5fSZe24HUc8ifmyZYOzVFG9om1lnbFvqc522LVq/fLCfnbllUubIlC6+7Bkd55XT6Ivg2nSuNflG0UNUdnAWM6yKDY68TpA31mk1No6dWm3372sucduV1APzcNxOAjeeWMu4Xibr7frw47O4/Rr7DC31usQbcN6Stzk5eTk+XTOox1qhcbpm1l63PZNxJFSd+oGY9VSqsqZkqUPuBm7RA8yv4dNbTLonHeol561Frctl9oA3JbllA2f05MBdWsAVqU403X2z8yeciQLksljwl/jxZ9sIv3jr7M3NRtO9MwMMXiwlp0BMHiFLSBzOjZMOTulLu83o8ATLY47CqsnhszHxagXOGz32gsJC4tVLn1LZiWi7oKvND8jaN+kl+/64FcpJC1vmpvJkt8v6r9/sA7HCL+Tyz594ANwSrL7rdTdKGzmQxz838bBo9P5ADo0uPkmm7zzVL2HGfjDcHB1iHNh8rJq7Xh7/qO5f2+W1yoHF3V5yv7x3b5z0Aziq6Vu47XFhl8QYBmeKtSHJohDMiLV386v3XGHehnFiivreCcBYv8bv22IE5z8j/uW3b8N4OccDfd4PMGxfd+2/fY4utHH5/3HgpAO1+zKmy0YCAaPKjx1lX9tRZBGPaMzAwMDAwMDCoJ5qNkaqaJ6Ni1tbq4NvpBdk9HvyN7GAXnPwvEq0d1AErq+kzN8pqPmrP4qDPbYxdRZU8GVYI7bguw3ClSXh8eaY4v6l5K4jfJyvtxPfktPK9k2W34d66ze9kbmVrdR4uIybPcnb+SsJ688UiSOLOcpx2riLbJBgdVXXnGGb5bNTkyFsBVrt5S0pImiu/R1q05OFJWbgLt51tOcT8HjWyYPVEZQdse1c0Ln04rgwxXznL/O8pGiA7Ynd2TrV18Z2D6NXsO0fymbxwsZynNzhanEaXlqb6dllpH4lJzO3VTaKndr6niQPH4uor/c3xqf/cqlfeFBNyj0fE2TrrYaln5r3z2Gg5796SMA2AHHdbSrSYvfpFSV/cVG6dyv6Wp8qJ9oH1CKqn9WzTkHQ10F0gmAM6oJes5qr1ktIiwyXlY1fF4c23UnW4ywkF4dbVKvJZ4+P4HseBlTKmQsZ424xpXfPkBZyEYN27e4zULdURjRfR+3u3irmM1Rurr7fWQVm4hqJq0E7AWXeVGUT8wUSJ70kKi3IrvN/53VI/y2Gluk5d66EgR0ygL046BYBrOoipL8bpxtlNWC13zi7fu5uiLxaPFleBMy7rRfSN4i6QZ2XlbgMkZkv9U16ToIGcD48C4J4eI8n/Qhjj9k75Lko5fe24sUzmR/dMGXd14faguugf98LHRFWW0dlWxtEJ/U7ElSqMvGf3Xl95n2tPJV3zFhTgsHInPnnHfwGRMdea889eJWeapv5RZHZv304w+Nux7kyUDcNIGRgYGBgYGBjUE7UyUkqpbsDrQGfAC0zRWj+tlEoF3gUygSzgEq11bqgvrtNqXqkqWbwDd4+ORGFkhk6WEOQURxxuazc7ZtodAGx87YVq312ii3CN/o7de93s2BRHuupFd3pRrstYxXyKKaKMEpRSKQ2R0edwrRSe/XI2kLJ8nwouHcX+4bIjyO0nTFTmh+Jno+PjcSQnWZWVZzicDqLzZRcZt0t2KosffFPKXANnHX+OlAOKvYdZlTeTTr0P4XAo3BsG0F31qSBfHPFo6pZVuqE7Mtvfbev9x/J1xuMAXL1ezvCKyd5VJaDA3skEe3eVNqRHUBmBit7DtaDanbVSeFKkTdq9JDu+57f9yEXLxefCYTn2eles9d9is4qWL58eNYAe14lv2MAo2fF1dIouj48v5anFwnK494pfSomnoFYZw6GndsJQR2wszo2ym2+3UB7nTE4m4z0JAaezsKm9XpOdXMGFx6GL5Hd64B/iexF9WJN7ichxcR/xm/lbB0lJ8s20VwJ8aAShtGM4ZAwZ9pl7lq9K24/dXJAgGaD3e6W/ZLy5DXel8wGr09USXcRqFjV+X7T7jifE8xkDzuS0HbbjM4QdLdVu3+kRZY9I4txvt8+o9t1Vx9OedKd3GPtipTpX/mzJ4etvg6ys+dafeswwn4eMM08Y5DbfbEBniGw/fSbJjF+aLI7Zs/rPYGLRqfJYh/yexe4iXMd/2+h6Gj1rkdT9hGF0e8Y6k+5HKy3QUf2I3yeS2Ol+uj1gpQ4aNYT8IvlcYKfYoYQC6/zED3PFv3PZfeJLzH1U7YuqFNeob2psx3DI6MmVW5XLhXuHdcqIrb8OZ/UnVShF/vuSYHN4jIxZeV4v5/0iflYpl1oJg9fPrfbdEB5/xVAYKTdwu9Z6ADAKuFEpNRC4G5itte4DzLb+bpFQKB6/vx2r52ZwrON0svVGDut8slhHKh0Zo8bjJApaqIwOHPRhCKvnZvDzjK5ks7mKfKl0pIyS5q5qvVGhDTmlWhmRDUGLRCgytmQ9hdYvo0IdWX1RnUq23mT6YgtEKO3Y0mUMF2plpLTWu4Bd1ucCpdRaIB04DzjZKvY/YA5wVzgqVZ3/VHUoGiuJ4/7T9TkAnCqKOcXiP9T7YUm8Ne7u6lebMSqOP4+XNAEuIJ4kSilmHzsZgUQrRBFNGcXnEwYZVXS0L5x0+MOSCA8N5anCMCm3NEtpZ2EnolV3sk8THxWvlWusMNNDzF7ZocTkir176KOTfWUK75RVfJ+bduACnBeN58Qb5d54nhT51C6OiT4DyiCNDDazpqGi+RDKKt8+FubDK58kzSkylL8oY2t0+bYq5Wt6VoU2VBCvq7ZhGhls4peUah9SR8yaIX5AE9dP9F1LbyPMkjtKfA+cqfI6XVaOipWd/p4LZYecN7aYD9JnAhBlMR9T80T+lx46n9T10h4ea0cWiozh0FPb78BbWMSXW8Rfz46g8xQUgOV357Aj1A7IjjIhK5sBP1j3FljpAFwunGXit/H6ueKj8rpbGNcB/zoIKkteqptWxkDUqKvW2FM8IhOA17o/RbxDwuMnbzoPAM+efVVuq05XY1QcMcQxrovoRTw/BdXThvZFO00H2us/NsSOgqzpLMqAsXZUF+mDiY4YfiixzutbL7v8GvuiI54/T5QQeheeoONpWPpi4JE7VXwZtc8C4MwVfW23SngD1y4/eWKfoectLsFxWHxrEnaKf2mPLyVitcP30bQrk3nE9uVrND2tlN7HlWkdxbNyCzPX/QDA0MdlnE/7IY/4udbZjgni16jXCFulnE6i5kh77znaSv/gVCRVok4mnnaxvG7LdtAVfVFjdHSTzIs+P0mPxxftOd5KOuo/kifIfUcP5KOjxLfU9ol+O78bbc4VH0av5a9YG9MUDl+3OjmbK6UygeHAAqCTtchCa71LKdUx1OfUZKKp7lp1cMTGcu9TrwL4Doz1aC93/UPyY7QrmFdrPQLPJSrWhRRwiDakUkYpMVYDOYS8C4uMutzNKdf+DoCOX4rDbrufUkiKkl3oXzp/DUDC1aL1P5em0sEpHT5JiXIkObx8WSgT8jPrxTmyaJ047bk7lBOTE1VRxi6gjh1McWmuyKfaUaZLiC6Td8SouJDPtKtWvoCBraY2tB131/5fOgB9o6JZVmY52m8XajfUmgSbBKtrwxgVBzp0na9NT31nB+aLqeundZlM6CgD7voXZEE04wcZzNr0zmV0WhYAY2O/AeDW1DU4rOrkWZT7k69L1t6Mbzbhyc+vKqP1Gxd7DzeannoDDmi185BhnSEYmCPMW1bJsVp78Rw85PtsI/EXWWjEDJeFc1k7mSTKOybh3GKlPckOvR3D2Reru2bDYS1+i2+RybejM8F3IPWGeZkA9HDvrlM97Mm+Jj1tcF+0TFC6XPsmJdvpXEVFV3Q8rwQ7j93NnV6yrkTxSJZsFlz7DtQuX5dhvoVxo/bFoE7RfhOlnVtK224UqbIxLR7QmbJkWbAkz90KQOlxfYg6LH3wkMR/0O0TKRO7twhtHd7b5H3RMuurhATG/mESAJ0/kzlDWe4D4A8qsgNFvEVFxO2XPvhtoRANqY4VLCqVMXfmp9Kve5ZYQTFef5BBsNxmTdEXVXQ0E4+SuUyXWua+qGi0NZbY+QQLz5G2nvrvJ2nnkHpsdkv7fDZxBLo8uFN5sHqMzxhZY18IFSErslIqEZgO3Kq1zrcPbA3hvknAJIBY4utTxyaDW7tZyTz6MQyXigp5Nm8pMro9pazY+C79VOuUD1p/G4KRsSa0FBnd2s1K/XOrlQ9afxuCkbEmtCQZG4qQFlJKqShkETVNa/2hdXmPUirNYqPSgL3B7tVaTwGmACSrVA3hodKc7SUNwObn0jkl7if7KgD/PDCQ9q/LqrqmNg90JPZqLyuZR2e601HJqj2aGEp1MTEqzg4bDY+M2kvs98Jc7J0k5o7Rid+TFi27+VhrkRrvkBX46XEFuCzZ7JOsS3U5RV5hdmKiZCdV2E1YA+fOWNqvlFW8nYE4Wf/EiiVT6UwnkU/rCvKV6mIUwRfHIctXU/bugLBqZ5o4Kc+54F9WoTgey54gH1duCFqHKrCeFZjwr8Y2dMRTqoshMBNbfWT03yBmLvz0+wtbe/HmQDm76upkCQ//96ViwnXjCWhD+Z2KtJu1ZUK73/DEnQB0WyAslM7L970qsB5er4eV/NxEeqrxllTy1amQIsCfgBFkx1h5d6fL3ZAnZj5PrMUQuOR/5dE+dtLHfFHStH2xGtjmBu8Qyfz9xeD/Wt/Es6Fc6t97qjBRnpr0PgC2rlZoQ4eE1Ufr8PbFwMAW2zxis2tERfkZKys7vx0A4erUgbU3y7iTYbVTnreETVuEYe1XHiREPDANgYXa2jCsfVFuqvK3clrpVawknY5DwtqU9Er0pyxpb5lYV+9CW2fMpc2XMyTzMkQHkhZs9yXiaOq+6C0Shl6VlRH3lYwJaogEtBRmJpK4zTIFWiZHm8lU/9/eucbGUV1x/Hd37dgueTgkcXCMkwAhIYiqECilEEpVaAspUloVJCqKEIrgS2kpaqnahg+0H2irCohQG6S2QRUVBChFEBqBKwi0AhoeCWleDuCUkJfjxnm/MPbu7YdzZzzeHe+O1+Pd9fj8JEve2Z3Z+9977p075557zwVzOd4idXp6WnS/3zuFjSelrzrzFef5PymeHFIGevPz2JajLfpbn2QyZN2Gx95GpNZaUuPEk9h1g2wF8eK9ct+Ykmqgx8q5N/9CFpWdvmPwGaggeRlVhknRYHMjrqcVQLu19sHAW6uAW93/twLPx1KiCmCtZSvvchoTmGXm+senMYNOJE6gV37wUakxir5OPqZGAgdHJVE1AocrU8Lhk3Q7heRr1LaobXG0MBY0xkUUj9QVwC3AJmOMN8H5c+DXwNPGmCXATuDGkSkikEr7ga2mVZZ5blsqTxJrL3+YFDJPesRtxPXEU1+hNfNW5Msf4QD72Ml4JrHWSnzSHC5gFvPYxFr22B1k6AXRHA8pGcNOf14yWq/iKj69TgKVD82RYNbvTZY8bQ1mnH9ad0Y0Hsymea5Tluk23i+fn+yN5rduwXjBh5/0cKh3X1F99TRQR31s8nwCT4resuqtS8UjNdVl6e61GdZtlafAuT3vFL1kWIxH0TrMikbcwok4ybpg68Z7Wnjlr9LheHnl0u5JP02KTM4WHllr+ecJebqc7jxRZrMEi+Z5gqiUnebktgzxtKS8zSqtzXs/1VDP9h9ILF9mutRZ47vy+fTJY3mehIpoDMFL3TP94R0ATEz1t43PpNzTf1cgP2Tu7+R5BnJs9YjtHqjPjnBbtBYzzuUgnSILVmxDHV5ohjnlPFcu5+eHd8zguS9Jmg0vvVZHXw91nTm3iqCn2XjP49lwjYRrZATaYijO+5adJH1iw/5e6na6uKkeF+dXW0P31eKtmfq21Ov4dudB/aTH1+h5Kg9H6FPjtFObyZB2i0DYKzGHqTMn+B611HHxMJ1Y0ArAhb98j7smyf3j3FqJa3vj1Gye2CBpY+a7PK5eTKOX+kpEyu8VpR5jbYtBW/LyrDZNof1HovGprz4M4MdFZbH89oB4lppelFi3UBdnoE/y4qziiIsKEmXV3uswiJ8Zro61NB6ecO8mNP40zGTp2Pomy6Dhxs/J1N0trVfw5C4Jvlt28PMAzHzpiB+gFoVGM5VruCH0PW91wlv2FY7agweHqCQca32XLS6IsWnlKcyz0nH9q15uxqsvkcC7hufepu9qCVquPeim7w4fp3a39EM24wUMSmPIABztnxqaSB3XmML6QDTGhbdqKGiwxgVCmh6p124XtHv7zIXMn+d2tI2w03NYI4hShwAv22fiSzDoypg9LlNXNd2HeOR33wRg820yIPxJ06sALJm5kOUfvw7ASy74c9nfr2fOY24F0TZJvpntG3yVStntFPIHUCblB5L7nVIwMbWXE9PdvFITxtPoZmv/1yzHxh0N1K3rML32MGIaC9lVSCJ074a55YBbLdsq5y2acSEf3S/T8XPSgZV1g6yEy7XVqHYaZ1v0pva86T47dRJmr9xI/amjZokXPutn/+aM74qWQ07Tm6fOYfbzrj9J5fyOxuRpr0hbzMWY/ikjF3Se3ikzUDX798N0Fx/d0D9gbfzA9ckuwT0usN5msv0DDae1EvcMb4dvb0BV33US3I76vWfLQo4jd0i4QfvFfTRvkoHho/sWArD1hXnMf0we8LzgdH8n/pA9ucqu0Zh++3K/967FTcyaLYt5pqW8VYVyH7m+5WK6vi8rgJuPbKAYJp3GpN0eftESEERGdzZXFEVRFEUpkYrl2iuINzp2wayZ4yd8D8vO28R1iYsxXv7x66w5JdN97cckIPKlFx7v3zPFDv+hp23vBi79+slhX6cQ2WPHwK06956eGwL52mrWrAf6n/5XB7PWx0DcGvO8Rsb4WeXn/lmm9m6/U56U0vPmyD4m4Hsh2/asi5aPLGKuMm+5a7p5KCqK4O167aa2Mt0HaPq9BONuXyVBxEt2icYDS77Itx+Qp6fGDnkc+vBPj3Dt0kuH9n0FdI6knXq70NsQj5nn6TA1Nf0ueefRWb2+zd/q44w1cmziExIQ2nflRaS8rRYiUrLGQvYR4k3q2yf1OOEBqcfr10h/0rHsMs57SHZ7z7osA23BthiTrcZaj86DmOl2Wxd0929h4HmOsx0yNbL3nsu5+QPpY6+aJgsm7p26jWe7xBuSycTTn0LMbTGXYG5LZ5Ne/2Pq6jD10ma9TBGr17dx3bwr5VSn0fPWperr8z2z5a5D+jN4ZJw3iXVb/CB4M0v61KbF4tnuWHYZH74nffDMlaJ/84rlXLvM7c/kLTIIehaHmO8xfjvt3/vrk6s+C0BDl6XuazsAqHW3iEUtC+T1a8186hy33qKVto4387fC8DzmmUxo/xXmkfaIqlE9UoqiKIqiKCVihrLx23CZaE63XzCFw6rainhaOh6SZdK/+sZKAM6p3c/yLsmDtPsyt8z6ywtIv7Z+4InDzLLu5oKLbp4Vh8aSKYPG4ejzll+bBhcs6LwR3sZ5cnDwJ15/B9ywp4qIT1Qv22fWWWsvKfSZWOowJ1bI9vWRdnEZmS6J1UhPndLvJYiJctppcPdscPWSY4OmpsavL09/9sDBQT8fhbJodPWXcos26O11xbR+zM2A+L9c26zytpiH01vT0syp+eLZr/2H7IiebpxE5vCRKMUeEuVqi8Gds+UfW9AL4Xk3BnhtqtVOc/DzC4Lv3fftNKyPHEX3xe9sk1ippzvFZE79poWGtRKA6dtniEZvq6RS+9ooGtUjpSiKoiiKUiJVFyM1mCfDWw4+527J/bXiblku/9GTiznrpo0DPl/zxub8jTiDo9QIo/A4MkIPxoh4o6CkOW6Ivzxh1zM1Nf1L+kOW9nukTpNVmdkTJ/LeC/VE+W/may97HQafhtyTbnDxqOeJ8l/H4I2qpJ2GLiEOxlwwsM5y9Q/4fAEqUo+u/rIFYriCK1PzbLOK2yKpdL4nxr3u272H2l0SA+bVYRzeqIraaVi/UcDz7W9mOuBgFdppCGFaC3pkqsROB7tmenqT32+sPG+GOyqeqbrabjK5fVCInqBu39sYVseOUjRW3UAqjOCeOrkicwdRIB1blESk1UrUihxJo46bYAMvVO7gACqSvmG6pmMlUIYk1mEuUdvYqNMYsezBgeSo0hgYRBRMED+a6zDAWG+LwYHEaNMYfPjKLftge0EVbLMFBlDDQaf2FEVRFEVRSqSswebGmP3ACaC72GergKkMLOcsa+20YicZY44B749YqeJlyBpHeR1C8jVGtdOxoFHbYvWgbXEQxojGZLfFcg6kAIwx7xZbqVENlFrO0aIPkq9xOOVUjdVD0u0Ukq9R7XTkzi0nSbdTKK2sOrWnKIqiKIpSIjqQUhRFURRFKZFKDKT+UIHvLIVSyzla9EHyNQ6nnKqxeki6nULyNaqdjty55STpdgollLXsMVKKoiiKoihJQaf2FEVRFEVRSqRsAyljzLXGmPeNMR3GmJ+W63uLYYxpNca8aoxpN8ZsMcbc5Y7fZ4zZY4zZ4P4WRbiWaqwQcWmsVn2QfI1qp6ox5zqJ1ufOUY0VIk6NWGtH/A9IA9uBs4FxwH+A88vx3RHK1gwscP9PAD4AzgfuA36sGseOxmrWNxY0qp2qxrGiTzUmR6O1tmweqUuBDmvtf621nwJPAovL9N0FsdZ2WmvXu/+PAe1ASwmXUo0VJCaNVasPkq9R7XRIJF1j0vWBaqwoMWos20CqBdgVeL2bEgs8khhjZgMXAW+5Q3caYzYaYx41xkwucrpqrBKGoXFU6IPka1Q7HfMak64PVGPVMEyNZRtImZBjVbVc0BgzHvgb8ENr7VHgEeAc4EKgE3ig2CVCjqnGMjNMjVWvD5KvUe1UNZJ8faAaq4IYNJZtILUbaA28PhPYW6bvLooxphb5IR+31j4LYK3tstZmrLVZ4I+Ii7IQqrHCxKCxqvVB8jWqnapGR9L1gWqsODFpLNtA6h3gXGPMWcaYccBNwKoyfXdBjDEGWAG0W2sfDBxvDnzsW8DmIpdSjRUkJo1Vqw+Sr1Ht1Ec1Jl8fqMaKEqPG8qzasxIVvwiJit8OLC3X90Yo10LE1bgR2OD+FgF/ATa546uAZtWYfI3Vqm8saFQ7VY1jSZ9qTI5G3dlcURRFURSlRHRnc0VRFEVRlBLRgZSiKIqiKEqJ6EBKURRFURSlRHQgpSiKoiiKUiI6kFIURVEURSkRHUgpiqIoiqKUiA6kFEVRFEVRSkQHUoqiKIqiKCXyfy2jkc7pPTauAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x144 with 20 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Compare original images with their reconstructions\n",
    "f, a = plt.subplots(2, 10, figsize=(10, 2))\n",
    "for i in range(examples_to_show):\n",
    "    a[0][i].imshow(np.reshape(mnist.test.images[i], (28, 28)))\n",
    "    a[1][i].imshow(np.reshape(encode_decode[i], (28, 28)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, the reconstructions were successful. It can be seen that some noise were added to the image."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Want to learn more?\n",
    "\n",
    "Running deep learning programs usually needs a high performance platform. __PowerAI__ speeds up deep learning and AI. Built on IBM’s Power Systems, __PowerAI__ is a scalable software platform that accelerates deep learning and AI with blazing performance for individual users or enterprises. The __PowerAI__ platform supports popular machine learning libraries and dependencies including TensorFlow, Caffe, Torch, and Theano. You can use [PowerAI on IMB Cloud](https://cocl.us/ML0120EN_PAI).\n",
    "\n",
    "Also, you can use __Watson Studio__ to run these notebooks faster with bigger datasets.__Watson Studio__ is IBM’s leading cloud solution for data scientists, built by data scientists. With Jupyter notebooks, RStudio, Apache Spark and popular libraries pre-packaged in the cloud, __Watson Studio__ enables data scientists to collaborate on their projects without having to install anything. Join the fast-growing community of __Watson Studio__ users today with a free account at [Watson Studio](https://cocl.us/ML0120EN_DSX).This is the end of this lesson. Thank you for reading this notebook, and good luck on your studies."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Thanks for completing this lesson!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Created by <a href=\"https://www.linkedin.com/in/franciscomagioli\">Francisco Magioli</a>, <a href=\"https://ca.linkedin.com/in/erich-natsubori-sato\">Erich Natsubori Sato</a>, <a href=\"https://ca.linkedin.com/in/saeedaghabozorgi\">Saeed Aghabozorgi</a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### References:\n",
    "- https://en.wikipedia.org/wiki/Autoencoder\n",
    "- http://ufldl.stanford.edu/tutorial/unsupervised/Autoencoders/\n",
    "- http://www.slideshare.net/billlangjun/simple-introduction-to-autoencoder\n",
    "- http://www.slideshare.net/danieljohnlewis/piotr-mirowski-review-autoencoders-deep-learning-ciuuk14\n",
    "- https://cs.stanford.edu/~quocle/tutorial2.pdf\n",
    "- https://gist.github.com/hussius/1534135a419bb0b957b9\n",
    "- http://www.deeplearningbook.org/contents/autoencoders.html\n",
    "- http://www.kdnuggets.com/2015/03/deep-learning-curse-dimensionality-autoencoders.html/\n",
    "- https://www.youtube.com/watch?v=xTU79Zs4XKY\n",
    "- http://www-personal.umich.edu/~jizhu/jizhu/wuke/Stone-AoS82.pdf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr>\n",
    "\n",
    "Copyright &copy; 2018 [Cognitive Class](https://cocl.us/DX0108EN_CC). This notebook and its source code are released under the terms of the [MIT License](https://bigdatauniversity.com/mit-license/)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  },
  "widgets": {
   "state": {},
   "version": "1.1.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}