tutorials/Part 09 - Converting the model to integers for FE.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Functional Encryption - Classification and information leakage\n",
    "\n",
    " \n",
    "### Encryption time!\n",
    "\n",
    "As we explained at the beginning of these tutorials, our quadratic network $Q$ is compatible with the Quadratic FE scheme detailed in [this paper](https://eprint.iacr.org/2018/206.pdf). But for this to be really effective, we need to convert the network weights to integers. We could use the fixed precision abstraction from the [PySyft](https://github.com/OpenMined/PySyft/) library which is perfect in this PyTorch context, but actually as we are not allowed truncation, we really need to work with real integers so it wouldn't be the most practical: therefore, we will write our own `fix_precision` function.\n",
    "\n",
    "In addition, note that all weights don't need the same precision as they have different amplitudes. We will assess how to scale the inputs and how to cap layers values (which are now integers) to have the smallest outputs possible, as without truncation the values will grow fast. We need the smallest output values possible beacuse we need to apply the discrete log to decode the values. Our objective is to to have an output which can be represented with $n$ bits, with $n$ sufficiently small to have all the exponent values in RAM memory so that the discrete log can be trivially implemented."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 9 Converting the model to integers\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load torch and syft packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Allow to load packages from parent\n",
    "import sys, os\n",
    "sys.path.insert(1, os.path.realpath(os.path.pardir))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from math import log2, ceil\n",
    "\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "import torch.utils.data as utils\n",
    "\n",
    "import learn\n",
    "from learn import collateral"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "PRIVATE_OUTPUT_SIZE = 4\n",
    "N_CHARS = 10\n",
    "N_FONTS = 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's load the quadratic model that we saved in Part 8! _Be sure that the path and file name match._"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "class CollateralNet(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(CollateralNet, self).__init__()\n",
    "        self.proj1 = nn.Linear(784, 40)\n",
    "        self.diag1 = nn.Linear(40, PRIVATE_OUTPUT_SIZE, bias=False)\n",
    "\n",
    "        # --- FFN for characters\n",
    "        self.lin1 = nn.Linear(PRIVATE_OUTPUT_SIZE, 32)\n",
    "        self.lin2 = nn.Linear(32, N_CHARS)\n",
    "\n",
    "        # --- Junction\n",
    "        self.jct = nn.Linear(PRIVATE_OUTPUT_SIZE, 784)\n",
    "\n",
    "        # --- CNN for families\n",
    "        self.conv1 = nn.Conv2d(1, 20, 5, 1)\n",
    "        self.conv2 = nn.Conv2d(20, 50, 5, 1)\n",
    "        self.fc1 = nn.Linear(4 * 4 * 50, 500)\n",
    "        self.fc2 = nn.Linear(500, N_FONTS)\n",
    "\n",
    "    def quad(self, x):\n",
    "        # --- Quadratic\n",
    "        x = x.view(-1, 784)\n",
    "        x = self.proj1(x)\n",
    "        x = x * x\n",
    "        x = self.diag1(x)\n",
    "        return x\n",
    "\n",
    "    def char_net(self, x):\n",
    "        # --- FFN\n",
    "        x = F.relu(x)\n",
    "        x = F.relu(self.lin1(x))\n",
    "        x = self.lin2(x)\n",
    "        return x\n",
    "\n",
    "    def font_net(self, x):\n",
    "        # --- Junction\n",
    "        x = self.jct(x)\n",
    "        x = x.view(-1, 1, 28, 28)\n",
    "\n",
    "        # --- CNN\n",
    "        x = F.relu(self.conv1(x))\n",
    "        x = F.max_pool2d(x, 2, 2)\n",
    "        x = F.relu(self.conv2(x))\n",
    "        x = F.max_pool2d(x, 2, 2)\n",
    "        x = x.view(-1, 4 * 4 * 50)\n",
    "        x = F.relu(self.fc1(x))\n",
    "        x = self.fc2(x)\n",
    "        return x\n",
    "\n",
    "    def forward_char(self, x):\n",
    "        x = self.quad(x)\n",
    "        x = self.char_net(x)\n",
    "        return F.log_softmax(x, dim=1)\n",
    "\n",
    "    def forward_font(self, x):\n",
    "        x = self.quad(x)\n",
    "        x = self.font_net(x)\n",
    "        return F.log_softmax(x, dim=1)\n",
    "    \n",
    "    # We add the ability to freeze some layers to ensure that the collateral task does\n",
    "    # not modify the quadratic net\n",
    "    \n",
    "    def get_params(self, net):\n",
    "        \"\"\"Select the params for a given part of the net\"\"\"\n",
    "        if net == 'quad':\n",
    "            layers = [self.proj1, self.diag1]\n",
    "        elif net == 'char':\n",
    "            layers = [self.lin1, self.lin2]\n",
    "        elif net == 'font':\n",
    "            layers = [self.jct, self.fc1, self.fc2, self.conv1, self.conv2]\n",
    "        else:\n",
    "            raise AttributeError(f'{net} type not recognized')\n",
    "        params = [p for layer in layers for p in layer.parameters()]\n",
    "        return params\n",
    "\n",
    "    def freeze(self, net):\n",
    "        \"\"\"Freeze a part of the net\"\"\"\n",
    "        net_params = self.get_params(net)\n",
    "        for param in net_params:\n",
    "            param.requires_grad = False\n",
    "\n",
    "    def unfreeze(self):\n",
    "        \"\"\"Unfreeze the net\"\"\"\n",
    "        for param in self.parameters():\n",
    "            param.requires_grad = True"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "CollateralNet(\n",
       "  (proj1): Linear(in_features=784, out_features=40, bias=True)\n",
       "  (diag1): Linear(in_features=40, out_features=4, bias=False)\n",
       "  (lin1): Linear(in_features=4, out_features=32, bias=True)\n",
       "  (lin2): Linear(in_features=32, out_features=10, bias=True)\n",
       "  (jct): Linear(in_features=4, out_features=784, bias=True)\n",
       "  (conv1): Conv2d(1, 20, kernel_size=(5, 5), stride=(1, 1))\n",
       "  (conv2): Conv2d(20, 50, kernel_size=(5, 5), stride=(1, 1))\n",
       "  (fc1): Linear(in_features=800, out_features=500, bias=True)\n",
       "  (fc2): Linear(in_features=500, out_features=2, bias=True)\n",
       ")"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "path = '../data/models/quadconvnet_part8.pt'\n",
    "model = CollateralNet()\n",
    "results = {}\n",
    "\n",
    "model.load_state_dict(torch.load(path))\n",
    "model.eval()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is a function to analyse a tensor distribution"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "def print_hist(data):\n",
    "    x = data.view(-1).abs()\n",
    "    # the histogram of the data\n",
    "    n, bins, patches = plt.hist(x, 50, density=True, facecolor='g', alpha=0.75)\n",
    "    plt.xlabel('Weight amplitude')\n",
    "    plt.ylabel('Probability')\n",
    "    plt.title('Weight amplitude distribution')\n",
    "    #plt.axis([40, 160, 0, 0.01])\n",
    "    plt.grid(True)\n",
    "    plt.show()\n",
    "    \n",
    "    \n",
    "def print_data(data):\n",
    "    ax = sns.heatmap(data, linewidth=0.5)\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And our function to convert tensors to integers with a precision parameter, and vice-versa"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def fix_precision(tensor, precision_bits, rm_outlier_frac=100, parameter=True):\n",
    "    tensor = (tensor * 2**precision_bits).long()\n",
    "    max_value = max(\n",
    "        np.abs(np.percentile(tensor, rm_outlier_frac)),\n",
    "        np.abs(np.percentile(tensor, 100 - rm_outlier_frac))\n",
    "    )\n",
    "    cp_tensor = 1 * tensor\n",
    "    tensor = tensor.clamp(min=-max_value, max=max_value)\n",
    "    if parameter:\n",
    "        return nn.Parameter(tensor, requires_grad=False)\n",
    "    else:\n",
    "        return tensor\n",
    "    \n",
    "def float_precision(tensor, precision_bits):\n",
    "    tensor = tensor.float()/2**precision_bits\n",
    "    return tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Parser:\n",
    "    \"\"\"Parameters for the testing\"\"\"\n",
    "    def __init__(self):\n",
    "        self.batch_size = 64\n",
    "        self.test_batch_size = 10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training set 60000 items\n",
      "Testing set  10000 items\n"
     ]
    }
   ],
   "source": [
    "args = Parser()\n",
    "\n",
    "data = learn.load_data()\n",
    "train_data, train_target_char, train_target_family, test_data, test_target_char, test_target_family = data\n",
    "test_target = test_target_char\n",
    "test_dataset = learn.build_tensor_dataset(test_data, test_target)\n",
    "test_loader = utils.DataLoader(\n",
    "    test_dataset,\n",
    "    batch_size=args.test_batch_size, shuffle=True\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our test function modifies the data first in the encrypted setting (ie values are integers) and then converts back the output to float to run the public part of the net used to predict digits."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "def test(model, test_loader, prec):\n",
    "    data_prec, proj_prec, diag_prec = prec \n",
    "    correct = 0\n",
    "    with torch.no_grad():\n",
    "        for data, target in test_loader:\n",
    "            # Private Part\n",
    "            data = fix_precision(data, data_prec, parameter=False)\n",
    "            private_output = model.quad(data)\n",
    "            # Public Part\n",
    "            output = float_precision(private_output, sum(prec))\n",
    "            output = model.char_net(output)\n",
    "\n",
    "            pred = output.argmax(1, keepdim=True)  # get the index of the max log-probability\n",
    "\n",
    "            correct += pred.eq(target.view_as(pred)).sum().item()\n",
    "\n",
    "    acc = 100. * correct / len(test_loader.dataset)\n",
    "    print('\\nTest set: Accuracy: {}/{} ({:.2f}%)\\n'.format(\n",
    "        correct, len(test_loader.dataset), acc))\n",
    "    \n",
    "    return acc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Given a precision setting, this returns the accuracy of the main task and the maximum number of bits needed to store the output."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "def test_compression(prec, rm_outlier_frac=100, show_distrib=False):\n",
    "    model = CollateralNet()\n",
    "    model.load_state_dict(torch.load(path))\n",
    "    model.eval()\n",
    "\n",
    "    data_prec, proj_prec, diag_prec = prec \n",
    "    \n",
    "    if show_distrib and False:\n",
    "        print_hist(model.proj1.weight.detach())\n",
    "        print_hist(model.proj1.bias.detach())\n",
    "        print_hist(model.diag1.weight.detach())\n",
    "        \n",
    "    # Convert the model\n",
    "    model.proj1.weight = fix_precision(model.proj1.weight, proj_prec, rm_outlier_frac)\n",
    "    model.proj1.bias = fix_precision(model.proj1.bias, proj_prec + data_prec, rm_outlier_frac)\n",
    "    model.diag1.weight = fix_precision(model.diag1.weight, diag_prec, rm_outlier_frac)\n",
    "        \n",
    "    if show_distrib:\n",
    "        print_hist(model.proj1.weight)\n",
    "        print_hist(model.proj1.bias)\n",
    "        print_hist(model.diag1.weight)\n",
    "    \n",
    "    data_sample = fix_precision(test_dataset[0][0], data_prec, parameter=False)\n",
    "    n_bits = (\n",
    "        ceil(log2(torch.max(data_sample))) * 2 +\n",
    "        ceil(log2(max(\n",
    "            torch.max(model.proj1.bias / 2 ** data_prec),\n",
    "            torch.max(model.proj1.weight)\n",
    "        ) * 2)) * 2 + \n",
    "        ceil(log2(torch.max(model.diag1.weight) * 2))   \n",
    "    )\n",
    "    print(\"approx size\", n_bits)\n",
    "        \n",
    "    test(model, test_loader, prec)\n",
    "    \n",
    "    return model, data_prec\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3X2cXGV9/vHPRRAFVoGARiUIqIkatIJZwIeKuy1q8AF+llSzFQQVU/trfMKHQluRRa3PKFUqRlC2glkpqE01FKhmQQQ1CaiYIGuMEcKDkASQRQsGvv3j3HM4zM7uzOzO2Zlkr/frta+dc+aee6452cx3zn3m3EcRgZmZGcBO7Q5gZmadw0XBzMxyLgpmZpZzUTAzs5yLgpmZ5VwUzMws56JgTZN0jqQPNtj2fEkfKTtTK0g6XdIF6fbTJI1ImtGivg+QFJJ2bkFfPZI2FZbXSuqZbL+przdKurywHJKe2Yq+U38jkp7eqv6s9VwUpgFJp0q6tGrdr8ZYt6hefxHx9oj4cIuytfRNp1Ui4uaI6IqIhwAkDUk6qd25aomIgyJiaLw2jRaliLgwIl7Rily1tlnaphta0b+Vw0VhergKeHHlU6+kpwCPAQ6pWvfM1NamoVbsxdj2z0VhelhFVgQOTssvBVYCN1Wt+3VE3AYg6dmSrpC0VdJNkl5f6ax6SEjSByTdLuk2SSfV+PS/l6TvSrpP0o8lPSM9rlKAfpaGFd5QHVzSMyR9X9IWSZslXShpz8L9GyW9X9LPJd0v6TxJsyRdmp7vfyTtldpWPi0vTllvl/S+Whus+Mla0kfT9vlCyvmFWp+8i5+MJc2Q9OmUeQPw6qr+90hZb5d0q6SPjDVUJWnXtM3vlrQOOLTq/o2Sjky3D5O0WtLvJf1O0pmpWWVb35New4sknSjph5I+K2kLcHpad3VVhFdJ2pBey6ck7ZSeKx9ua2SbpTb530baBv8u6S5Jv5X0z4W+T5R0ddqGd0v6jaSjam0fay0XhWkgIh4EfgwckVYdAfwAuLpq3VUAknYHrgC+DjwJWAT8m6R51X1LWgCcDBxJtqfRUyPCIqAf2AtYD3w05ao89/PTsMI3ajxWwMeApwLPAfYDTq9qcyzwcmAu8FrgUuAfgSeS/Y2/s6p9LzAHeAXwD5U31LFExD+Rba8lKeeS8donbwNeAxwCdAMLq+4/H9hGts0OSVnGGp76EPCM9PNK4IRxnvcs4KyIeEJqf1FaX9nWe6bXcG1aPhzYAMwi/bvU8Lr0Gl4AHAO8ZZznBxreZp8H9gCeDrwMeBPw5sL9h5N9cNkH+CRwniTVe26bHBeF6eNKHnljeCnZf9gfVK27Mt1+DbAxIr4aEdsi4nrgEuCva/T7euCrEbE2Iv7A6DdsgG9FxE8iYhtwIY/sndQVEesj4oqIeCAi7gLOJHsDKfp8RPwuIm5Nr+nHEXF9RPwv8C2yN92i/oi4PyJuAL4K9DWapwmvBz4XEbdExFaywgaApFnAq4B3pxx3Ap8lK55j9fXRiNgaEbcA/zrO8/4JeKakfSJiJCJ+VCfnbRHx+fTv/Mcx2nwiPffNwOdowfZKe0WLgFMj4r6I2Ah8Bji+0Oy3EfHldFxnAHgKWfGyErkoTB9XAX8uaSbwxIj4FXAN2bGGmcBzeWSIYX/gcEn3VH6ANwJPrtHvU4FbCsu31GhzR+H2H4CuRkOnoaDBNMTye+ACsk+ORb8r3P5jjeXq5ytm/C3Za2i16u3y28Lt/cmG824vbN8vke2VNdtXtbeS7TH9UtIqSa+pk7PWv9d4bVq1vfYh2wbF1/JbYN/Ccv53kz5wQBN/OzYxLgrTx7Vku+pvA34IEBG/B25L626LiN+ktrcAV0bEnoWfroj4uxr93g7MLizv1+Lc/wIE8Lw0JHIc2ZDSZBQzPo1sG9RTPZ3w/en3boV1xaJ5e43nqbgFeADYp7B9nxARB43x3OP19eiQEb+KiD6yAvMJ4OI0HDjWdMiNTJM81va6n7Fff72+N5Pt1exf1fetDeSxErkoTBNpaGA12fj/Dwp3XZ3WFb919B1grqTjJT0m/Rwq6Tk1ur4IeLOk50jaDWjo/IWC35GNKY/l8cAIcK+kfYH3N9l/LR+UtJukg8jGsGsdy6j2qJxpKOtW4Lh0UPktZGP4FRcB75Q0Ox3oPqXw2NuBy4HPSHqCpJ2UHVCvHhYr9nWqpL0kzQbeMVZIScdJemJEPAzck1Y/DNyVfk/kHIH3p+feD3gXj2yvnwJHKDunYw/g1KrHjflvm4aELgI+KunxkvYn+zu8oFZ7mzouCtPLlWSfIIvfLvlBWpcXhYi4j+zA5yKyT4V3kH3qfGx1hxFxKdkY90qyg8iVMewHGsx0OjCQhlFeX+P+frIDnPcC3wW+2WC/47mSLOv3gE9HxOV12kN2AHdh+iZMZUz/bWRFagtwENlwXMWXgcuAnwHX1cj9JmAXYB1wN3Ax2Zh5Lf1kQyu/ISsmXxsn5wJgraSRlHlRRPwxDb98FPhh2tYvrP+Sc/8JrCErAt8FzgOIiCvICsTP0/3fqXpcrW1W9A6yvY0NZH+TXwe+0kQuK4F8kR1rpbQ38QvgsenAcseQdADZG+tjOi2bWafwnoJNmqTXSXpsGib5BPBfftM12z65KFgr/C1wJ/Br4CGg1gFpM9sOePjIzMxy3lMwM7PcdjcB1j777BMHHHDAhB57//33s/vuu7c2UAs4V3Ocq3mdms25mjOZXGvWrNkcEU+s2zAitquf+fPnx0StXLlywo8tk3M1x7ma16nZnKs5k8kFrI4G3mM9fGRmZjkXBTMzy7komJlZzkXBzMxyLgpmZpZzUTAzs5yLgpmZ5VwUzMws56JgZma57W6ai8kY3jJM/0D/qPUrT1jZhjRmZp1nWhWFVuod6K253gXGzLZnHj4yM7Oci4KZmeVKLQqSFki6SdJ6SaeM0eb1ktZJWivp62XmMTOz8ZV2TEHSDOBs4OXAJmCVpOURsa7QZg5wKvCSiLhb0pPKymNmZvWVeaD5MGB9RGwAkDQIHAOsK7R5G3B2RNwNEBF3lpinI/UO9NLX1TfqW1E+YG1m7VDaNZolLQQWRMRJafl44PCIWFJo821gGHgJMAM4PSL+u0Zfi4HFALNmzZo/ODg4oUyb79nM1oe2jlo/d++5Tfc1vGW45vpm+xreMszMGTNH5ZpIplYbGRmhq6ur3TFGca7mdWo252rOZHL19vauiYjueu3a/ZXUnYE5QA8wG7hK0vMi4p5io4hYCiwF6O7ujp6engk92dJLlrJsZNmo9SuPbf5Tea3zHSbSV/9AP31dfaNyTSRTqw0NDTHRbV0m52pep2ZzruZMRa4yDzTfCuxXWJ6d1hVtApZHxJ8i4jdkew1zSsxkZmbjKLMorALmSDpQ0i7AImB5VZtvk+0lIGkfYC6wocRMZmY2jtKKQkRsA5YAlwE3AhdFxFpJZ0g6OjW7DNgiaR2wEnh/RGwpK5OZmY2v1GMKEbECWFG17rTC7QBOTj9mZtZmPqPZzMxyLgpmZpZzUTAzs5yLgpmZ5VwUzMws56JgZma5dk9zYS3iK8GZWSt4T8HMzHIuCmZmlnNRMDOznIuCmZnlXBTMzCznomBmZjkXBTMzy7komJlZzkXBzMxyLgpmZpZzUTAzs5yLgpmZ5VwUzMws56JgZmY5FwUzM8u5KJiZWc5FwczMcqUWBUkLJN0kab2kU2rcf6KkuyT9NP2cVGYeMzMbX2mX45Q0AzgbeDmwCVglaXlErKtq+o2IWFJWDjMza1yZewqHAesjYkNEPAgMAseU+HxmZjZJiohyOpYWAgsi4qS0fDxweHGvQNKJwMeAu4Bh4D0RcUuNvhYDiwFmzZo1f3BwcEKZNt+zma0PbR21fu7ec5vua3jLcM31zfY1vGWYmTNmjso1kX5akadoZGSErq6uCT++LM7VvE7N5lzNmUyu3t7eNRHRXa9du4vC3sBIRDwg6W+BN0TEX4zXb3d3d6xevXpCmZZespRlI8tGrV95wsqm++od6K25vtm+egd66evqG5VrIv20Ik/R0NAQPT09E358WZyreZ2azbmaM5lckhoqCmUOH90K7FdYnp3W5SJiS0Q8kBbPBeaXmMfMzOoosyisAuZIOlDSLsAiYHmxgaSnFBaPBm4sMY+ZmdVR2rePImKbpCXAZcAM4CsRsVbSGcDqiFgOvFPS0cA2YCtwYll5zMysvtKKAkBErABWVK07rXD7VODUMjOYmVnjfEazmZnlXBTMzCznomBmZjkXBTMzy7komJlZzkXBzMxyLgpmZpZzUTAzs5yLgpmZ5VwUzMws56JgZmY5FwUzM8u5KJiZWc5FwczMci4KZmaWc1EwM7Oci4KZmeVcFMzMLOeiYGZmORcFMzPLuSiYmVnORcHMzHIuCmZmlnNRMDOzXKlFQdICSTdJWi/plHHaHSspJHWXmcfMzMZXWlGQNAM4GzgKmAf0SZpXo93jgXcBPy4ri5mZNabMPYXDgPURsSEiHgQGgWNqtPsw8Angf0vMYmZmDSizKOwL3FJY3pTW5SS9ANgvIr5bYg4zM2uQIqJ+I+mbwHnApRHxcEMdSwuBBRFxUlo+Hjg8Ipak5Z2A7wMnRsRGSUPA+yJidY2+FgOLAWbNmjV/cHCwkQijbL5nM1sf2jpq/dy95zbd1/CW4Zrrm+1reMswM2fMHJVrIv20Ik/RyMgIXV1dE358WZyreZ2azbmaM5lcvb29ayKi7nHbRovCkcCbgRcC/wF8NSJuqvOYFwGnR8Qr0/KpABHxsbS8B/BrYCQ95MnAVuDoWoWhoru7O1avHvPucS29ZCnLRpaNWr/yhJVN99U70FtzfbN99Q700tfVNyrXRPppRZ6ioaEhenp6Jvz4sjhX8zo1m3M1ZzK5JDVUFBoaPoqI/4mINwIvADYC/yPpGklvlvSYMR62Cpgj6UBJuwCLgOWFPu+NiH0i4oCIOAD4EXUKgpmZlavhYwqS9gZOBE4CrgfOIisSV9RqHxHbgCXAZcCNwEURsVbSGZKOnmRuMzMrwc6NNJL0LeBZwNeA10bE7emub0ga85N9RKwAVlStO22Mtj2NZDEzs/I0VBSAL6c3+Jykx0bEA42MUZmZ2fah0eGjj9RYd20rg5iZWfuNu6cg6clk5xbsKukQQOmuJwC7lZzNzMymWL3ho1eSHVyeDZxZWH8f8I8lZTIzszYZtyhExAAwIOnYiLhkijKZmVmb1Bs+Oi4iLgAOkHRy9f0RcWaNh5mZ2Xaq3vDR7ul3553vbWZmLVdv+OhL6Xf/1MQxM7N2qjd89K/j3R8R72xtHDMza6d6w0drpiSFmZl1hEa+fWRmZtNEveGjz0XEuyX9FzBqju2I8MR2O5jiFNx9XX30D2SHkyYzBbeZbT/qDR99Lf3+dNlBzMys/eoNH61Jv69M10R4Ntkew03pustmZrYDaXTq7FcD55BdKU3AgZL+NiIuLTOcmZlNrUanzv4M0BsR6wEkPQP4LuCiYGa2A2l06uz7KgUh2UA2KZ6Zme1A6n376K/SzdWSVgAXkR1T+GuyazCbmdkOpN7w0WsLt38HvCzdvgvYtZREZmbWNvW+ffTmqQpiZmbt1+i3jx4HvBU4CHhcZX1EvKWkXGZm1gaNHmj+GvBksiuxXUl2JTYfaDYz28E0WhSeGREfBO5P8yG9Gji8vFhmZtYOjRaFP6Xf90h6LrAH8KRyIpmZWbs0evLaUkl7AR8ElpNdie2DpaUyM7O2aGhPISLOjYi7I+LKiHh6RDypclW28UhaIOkmSeslnVLj/rdLukHSTyVdLWneRF6EmZm1RkNFQdLekj4v6TpJayR9TtLedR4zAzgbOAqYB/TVeNP/ekQ8LyIOBj4JnDmB12BmZi3S6DGFQeBO4FhgIbAZ+EadxxwGrI+IDWlG1UHgmGKDiPh9YXF3alyzwczMpo4i6r8PS/pFRDy3at0NEfG8cR6zEFgQESel5eOBwyNiSVW7vwdOBnYB/iIiflWjr8XAYoBZs2bNHxwcrJu5ls33bGbrQ1tHrZ+799ym+xreMlxzfbN9DW8ZZuaMmaNyTaSfVuWpKOaayDYqy8jICF1dXe2OMUqn5oLOzeZczZlMrt7e3jUR0V2vXaNF4UzgJ2RzH0G2t3BYRLxvnMc0VBQK7f8GeGVEnDBelu7u7li9enXdzLUsvWQpy0aWjVo/kauKFa9QNpm+egd66evqG5VrIv20Kk9FMVcnXXltaGiInp6edscYpVNzQedmc67mTCaXpIaKQr0J8e4jG9IR8G7ggnTXTsAIMGZRAG4F9issz07rxjIIfLFOXjMzK1G9uY8eP4m+VwFzJB1IVgwWAX9TbCBpTmG46NXAqKEjMzObOo2ep4Cko4Ej0uJQRHxnvPYRsU3SEuAyYAbwlYhYK+kMYHVELAeWSDqS7OS4u4Fxh47MzKxcjU6I93HgUODCtOpdkl4SEaeO97iIWAGsqFp3WuH2u5qLa2ZmZWp0T+FVwMER8TCApAHgemDcomBmZtuXRs9TANizcHuPVgcxM7P2a3RP4WPA9ZJWkn0T6Qhg1LQVZma2fatbFCQJuBp4IdlxBYB/iIg7ygxmZmZTr25RiIiQtCKdvbx8CjKZmVmbNHpM4TpJh9ZvZmZm27NGjykcDhwnaSNwP9lxhYiIPysrmJmZTb1Gi8IrS01hZmYdod7cR48D3g48E7gBOC8itk1FMDMzm3r1jikMAN1kBeEo4DOlJzIzs7apN3w0r3LNBEnnkU2fbWZmO6h6ewp/qtzwsJGZ2Y6v3p7C8yVVLpkpYNe0XPn20RNKTWdmZlOq3vUUZkxVEDMza79mJsQzM7MdnIuCmZnlXBTMzCznomBmZjkXBTMzy7komJlZzkXBzMxyLgpmZpZzUTAzs5yLgpmZ5UotCpIWSLpJ0npJp9S4/2RJ6yT9XNL3JO1fZh4zMxtfaUVB0gzgbLLrMMwD+iTNq2p2PdCdLut5MfDJsvKYmVl9Ze4pHAasj4gNEfEgMAgcU2wQESsj4g9p8UfA7BLzmJlZHYqIcjqWFgILIuKktHw8cHhELBmj/ReAOyLiIzXuWwwsBpg1a9b8wcHBCWXafM9mtj60ddT6uXvPbbqv4S3DNdc329fwlmFmzpg5KtdE+mlVnopirolso7KMjIzQ1dXV7hijdGou6NxsztWcyeTq7e1dExHd9drVu57ClJB0HNllP19W6/6IWAosBeju7o6enp4JPc/SS5aybGTZqPUrj13ZdF/9A/011zfbV/9AP31dfaNyTaSfVuWpKOaayDYqy9DQEBP9GyhTp+aCzs3mXM2ZilxlFoVbgf0Ky7PTukeRdCTwT8DLIuKBEvOYmVkdZR5TWAXMkXSgpF2ARcDyYgNJhwBfAo6OiDtLzGJmZg0orSikazovAS4DbgQuioi1ks6QdHRq9imgC/gPST+VtHyM7szMbAqUekwhIlYAK6rWnVa4fWSZz29mZs3xGc1mZpZzUTAzs5yLgpmZ5VwUzMws56JgZmY5FwUzM8u5KJiZWc5FwczMci4KZmaWc1EwM7Oci4KZmeVcFMzMLOeiYGZmORcFMzPLuSiYmVnORcHMzHIuCmZmlnNRMDOznIuCmZnlXBTMzCy3c7sD2I6rd6C35vqVJ6yc4iRm1ijvKZiZWc5FwczMci4KZmaWK7UoSFog6SZJ6yWdUuP+IyRdJ2mbpIVlZjEzs/pKKwqSZgBnA0cB84A+SfOqmt0MnAh8vawcZmbWuDK/fXQYsD4iNgBIGgSOAdZVGkTExnTfwyXmMDOzBikiyuk4Gw5aEBEnpeXjgcMjYkmNtucD34mIi8foazGwGGDWrFnzBwcHJ5Rp8z2b2frQ1lHr5+49t+m+hrcM11zfbF/DW4aZOWPmqFwT6adVeSqKudq5jaqNjIzQ1dU1qT7K0Km5oHOzOVdzJpOrt7d3TUR012u3XZynEBFLgaUA3d3d0dPTM6F+ll6ylGUjy0atX3ls89+b7x/or7m+2b76B/rp6+oblWsi/bQqT0UxVzu3UbWhoSEm+jdQpk7NBZ2bzbmaMxW5yjzQfCuwX2F5dlpnZmYdqsyisAqYI+lASbsAi4DlJT6fmZlNUmlFISK2AUuAy4AbgYsiYq2kMyQdDSDpUEmbgL8GviRpbVl5zMysvlKPKUTECmBF1brTCrdXkQ0rmZlZB/AZzWZmlnNRMDOznIuCmZnltovzFGx6q74uQ19XH/0D/b4ug1kJvKdgZmY5FwUzM8u5KJiZWc5FwczMci4KZmaWc1EwM7Ocv5Jq00b1V1sr/NVWs0d4T8HMzHIuCmZmlnNRMDOznIuCmZnlXBTMzCznbx+ZtcjwlmH6B/pr3udvONn2wnsKZmaWc1EwM7Oci4KZmeV8TMGsA/nsa2sX7ymYmVnOewpmO7DKHkflEqYV3uOwsbgomFldHs6aPjx8ZGZmuVL3FCQtAM4CZgDnRsTHq+5/LPDvwHxgC/CGiNhYZiYza6/iXkdxWMt7HZ2htKIgaQZwNvByYBOwStLyiFhXaPZW4O6IeKakRcAngDeUlcnMdhytGtKq1U9fVx899EwkVkuM9do+tP+HSn/uMvcUDgPWR8QGAEmDwDFAsSgcA5yebl8MfEGSIiJKzGVm1nI7ynEXlfX+K2khsCAiTkrLxwOHR8SSQptfpDab0vKvU5vNVX0tBhanxWcBN00w1j7A5rqtpp5zNce5mtep2ZyrOZPJtX9EPLFeo+3i20cRsRRYOtl+JK2OiO4WRGop52qOczWvU7M5V3OmIleZ3z66FdivsDw7ravZRtLOwB5kB5zNzKwNyiwKq4A5kg6UtAuwCFhe1WY5cEK6vRD4vo8nmJm1T2nDRxGxTdIS4DKyr6R+JSLWSjoDWB0Ry4HzgK9JWg9sJSscZZr0EFRJnKs5ztW8Ts3mXM0pPVdpB5rNzGz74zOazcws56JgZma5aVMUJC2QdJOk9ZJOaXceAEn7SVopaZ2ktZLe1e5MRZJmSLpe0nfanaVC0p6SLpb0S0k3SnpRuzMBSHpP+jf8haRlkh7XphxfkXRnOgeosm6mpCsk/Sr93qtDcn0q/Tv+XNK3JO3ZCbkK971XUkjap1NySXpH2mZrJX2yjOeeFkWhMOXGUcA8oE/SvPamAmAb8N6ImAe8EPj7DslV8S7gxnaHqHIW8N8R8Wzg+XRAPkn7Au8EuiPiuWRfrCj7SxNjOR9YULXuFOB7ETEH+F5anmrnMzrXFcBzI+LPgGHg1KkORe1cSNoPeAVw81QHSs6nKpekXrJZIJ4fEQcBny7jiadFUaAw5UZEPAhUptxoq4i4PSKuS7fvI3uD27e9qTKSZgOvBs5td5YKSXsAR5B9a42IeDAi7mlvqtzOwK7pfJvdgNvaESIiriL7Jl/RMcBAuj0A/L8pDUXtXBFxeURsS4s/IjuXqe25ks8CHwDa8k2cMXL9HfDxiHggtbmzjOeeLkVhX+CWwvImOuTNt0LSAcAhwI/bmyT3ObL/FA+3O0jBgcBdwFfTsNa5knZvd6iIuJXsU9vNwO3AvRFxeXtTPcqsiLg93b4DmNXOMGN4C3Bpu0MASDoGuDUiftbuLFXmAi+V9GNJV0o6tIwnmS5FoaNJ6gIuAd4dEb/vgDyvAe6MiDXtzlJlZ+AFwBcj4hDgftozFPIoaYz+GLKi9VRgd0nHtTdVbenk0I76HrqkfyIbSr2wA7LsBvwjcFq7s9SwMzCTbKj5/cBFktTqJ5kuRaGRKTfaQtJjyArChRHxzXbnSV4CHC1pI9lQ219IuqC9kYBsD29TRFT2pi4mKxLtdiTwm4i4KyL+BHwTeHGbMxX9TtJTANLvUoYdJkLSicBrgDd2yGwGzyAr7j9Lf/+zgeskPbmtqTKbgG9G5idke/EtPwg+XYpCI1NuTLlU5c8DboyIM9udpyIiTo2I2RFxANm2+n5EtP2Tb0TcAdwi6Vlp1V/y6KnY2+Vm4IWSdkv/pn9JBxwALyhOJ3MC8J9tzJJLF+H6AHB0RPyh3XkAIuKGiHhSRByQ/v43AS9If3vt9m2gF0DSXGAXSpjJdVoUhXQwqzLlxo3ARRGxtr2pgOwT+fFkn8R/mn5e1e5QHe4dwIWSfg4cDPxLm/OQ9lwuBq4DbiD7f9WWaRIkLQOuBZ4laZOktwIfB14u6VdkezUfH6+PKcz1BeDxwBXpb/+cDsnVdmPk+grw9PQ11UHghDL2rjzNhZmZ5abFnoKZmTXGRcHMzHIuCmZmlnNRMDOznIuCmZnlXBSsY0n6rKR3F5Yvk3RuYfkzkk6u08c1DTzPxlozYUrqkTSlJ6FJOlHSF9Ltt0t6U2H9UyfQX83XZjYWFwXrZD8knRksaSeyszcPKtz/YmDcN/2ImMybeg9tPDM5Is6JiH9PiyeSTaFhVioXBetk1wCV6yUcBPwCuE/SXpIeCzyH7IQxJL1f0qo0N39/pQNJI+n3TpL+Lc1Ff4WkFZIWFp7rHZKuk3SDpGenCQrfDrwnnVj10mIwSYdJujZNzHdN5Szr9In+2+k5NkpaIunk1O5HkmamdkOSzkp9/0LSYdUvXtLpkt6XcnaTnbT3U0m7FvcAJHVLGkq395Z0ubL59s8FVOjvOEk/SX18SdmU8maP4qJgHSsibgO2SXoa2Sf2a8lmkX0R2ZvkDRHxoKRXAHPIpkg/GJgv6Yiq7v4KOIDsehrH80ixqdgcES8Avgi8LyI2AucAn42IgyPiB1Xtfwm8NE3MdxqPPrP6uen5DgU+CvwhtbsWeFOh3W4RcTDw/8nOVh1rO1wMrCabH+jgiPjjWG2BDwFXp/n2vwU8DUDSc4A3AC9Jz/kQ8MZx+rFpaud2BzCr4xqygvBi4EyyKc9fDNxLNrwE2cVQXgFcn5a7yIrEVYV+/hz4j4h4GLhD0sqq56lMRriG7A29nj2AAUlzyGYdfUzhvpXp+hj3SboX+K+0/gbgzwrtlkE2d76kJ6g1Vx47gpQ/Ir4r6e60/i+B+cCqNLHmrnTQxHjWOVwUrNNVjis8j2z46BbgvcDvga+mNgI+FhFfmsTzPJB+P0Rj/y8+TPbm/7o01DRUoy/IZrJ8oHC72Hf1HDOP2Xw+AAABVklEQVTNzDmzjUf29Bu59KeAgYhox9XNbDvi4SPrdNeQTa28NSIeioitwJ5kwz+Vg8yXAW9Rdl0KJO0r6UlV/fwQODYdW5hFdhC5nvvIJmyrZQ8emX79xAZfS7U3AEj6c7IL89zbRJaNZJ/8AY4trL8K+JvU71FA5XrM3wMWVraLsus27z/B3LYDc1GwTncD2beOflS17t6I2AzZZR2BrwPXSrqBbMbS6jfzS8imQV4HXEB2gHq8N2HIhn1eV+tAM/BJ4GOSrmfie9z/mx5/DlBvds7zgXMqB5qBfuAsSavJ9m4q+oEjJK0lG0a6GSAi1gH/DFyeZpi9AnjKBHPbDsyzpNq0IakrIkYk7Q38hOyga1vmyU/fFnpfRKxux/ObjcXHFGw6+U46mLsL8OEOuXCKWUfxnoKZmeV8TMHMzHIuCmZmlnNRMDOznIuCmZnlXBTMzCz3f9ENZDNnAJhNAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEWCAYAAACXGLsWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3X18XVWd7/HPlwIViBRasCIFilKUoiNIpMook4hgUbQ6FGgFLAr2cp36MAzOgFcZijAOXgVRcBAtmgFt4ILMVIELKA3yJNICCgWpoRQpT9KWp6CAhd/8sVfg9HCSdZJmJyfJ9/165ZW911577d9Z3T2/7LXPWVsRgZmZWW82GuoAzMys8TlZmJlZlpOFmZllOVmYmVmWk4WZmWU5WZiZWZaThQ0oSedK+kqddX8k6dSyYxoIkk6WdGFa3lFSl6QxA9T2ZEkhaeMBaKtF0qqK9WWSWja03dTW4ZKurlgPSbsMRNupvS5Jbxyo9mxgOVmMcpJOlHRlVdkfeiiblWsvIo6NiK8OUGwD+mY0UCLijxHRFBEvAkjqkHTMUMdVS0TsHhEdvdWpN1lFxI8j4oCBiKtWn6U+XTEQ7dvAc7KwXwH7dP+VLGk7YBNgz6qyXVJdG4UG4qrHhjcnC7uVIjnskdbfCywG7q0quy8iHgaQ9BZJ10haK+leSYd2N1Y9tCTpnyU9IulhScfUuFrYWtLlkp6RdIukN6X9uhPTb9PwxGHVgUt6k6RrJa2RtFrSjyVtVbF9paQvSvqdpGclLZA0UdKV6Xi/kLR1qtv91/XcFOsjko6v1WGVf4lLOi31z9kpzrNr/aVe+Ze0pDGSvpFiXgF8qKr9cSnWRyQ9JOnUnoa8JG2W+vwJSXcD76zavlLS+9Py3pKWSHpa0mOSzkjVuvv6yfQa3i3pKEk3SjpT0hrg5FR2Q1UIH5S0Ir2W/ytpo3Ssl4ft6umzVOflcyP1wX9KelzSA5K+XNH2UZJuSH34hKT7JR1Yq39s4DhZjHIR8QJwC7BvKtoXuB64oarsVwCStgCuAX4CvA6YBXxX0tTqtiVNB44D3k9xZdJSI4RZwHxga6ATOC3F1X3st6fhiYtq7Cvga8AbgN2AHYCTq+ocDOwP7Ap8GLgS+BKwLcX5/7mq+q3AFOAA4F+632h7EhH/h6K/5qU45/VWP/k0cBCwJ9AMzKza/iNgHUWf7Zli6WmY61+BN6WfDwBzejnuWcBZEbFlqn9xKu/u663Sa7g5rU8DVgATSf8uNXwsvYZ3ADOAT/VyfKDuPvsOMA54I/B3wCeAT1Zsn0bxB802wNeBBZKUO7b1n5OFAVzHK28Y76X4j3x9Vdl1afkgYGVE/DAi1kXE7cClwCE12j0U+GFELIuIP/PqN3KAyyLiNxGxDvgxr1zNZEVEZ0RcExHPR8TjwBkUbyyVvhMRj0XEQ+k13RIRt0fEc8BlFG/GleZHxLMRcSfwQ2B2vfH0waHAtyLiwYhYS5HwAJA0Efgg8IUUx5+AMymSak9tnRYRayPiQeDbvRz3r8AukraJiK6I+HUmzocj4jvp3/kvPdQ5PR37j8C3GID+SldRs4ATI+KZiFgJfBM4sqLaAxHx/XTfqA3YjiKpWUmcLAyKq4b3SBoPbBsRfwBuoriXMR54K68MVewETJP0ZPcPcDjw+hrtvgF4sGL9wRp1Hq1Y/jPQVG/QaUipPQ3VPA1cSPGXZqXHKpb/UmO9+niVMT5A8RoGWnW/PFCxvBPFsOAjFf37PYqruL62Ve1oiius30u6VdJBmThr/Xv1Vmeg+msbij6ofC0PANtXrL983qQ/RKAP5471nZOFAdxMccn/aeBGgIh4Gng4lT0cEfenug8C10XEVhU/TRHxv2u0+wgwqWJ9hwGO+9+AAN6WhlaOoBia2hCVMe5I0Qc51VM3P5t+b15RVplMH6lxnG4PAs8D21T075YRsXsPx+6trfWDjPhDRMymSDynA5ekYcWepp6uZ0rqnvrrWXp+/bm2V1NcBe1U1fZDdcRjJXGyMNIQwxKK+wvXV2y6IZVVfgrq58Cuko6UtEn6eaek3Wo0fTHwSUm7SdocqOv7FxUeoxiz7slrgS7gKUnbA1/sY/u1fEXS5pJ2pxgjr3WvpNp6caYhsYeAI9LN7E9R3CPodjHwOUmT0g32Eyr2fQS4GvimpC0lbaTiRn718FplWydK2lrSJOCzPQUp6QhJ20bES8CTqfgl4PH0uz/fcfhiOvYOwOd5pb/uAPZV8Z2UccCJVfv1+G+bhpYuBk6T9FpJO1GchxfWqm+Dw8nCul1H8Rdn5addrk9lLyeLiHiG4obrLIq/Ih+l+Ct1bHWDEXElxRj6Yoqb191j5M/XGdPJQFsajjm0xvb5FDdWnwIuB35aZ7u9uY4i1l8C34iIqzP1obhxPDN9Mqf7nsGnKZLXGmB3imG9bt8HrgJ+C9xWI+5PAJsCdwNPAJdQjMnXMp9iiOZ+iiRzQS9xTgeWSepKMc+KiL+kYZzTgBtTX78r/5Jf9t/AUorkcDmwACAirqFIHL9L239etV+tPqv0WYqrkxUU5+RPgPP7EJcNMPnhRzZY0tXHXcDYdEO7YUiaTPGGu0mjxWbWCHxlYaWS9DFJY9Nwy+nAz/xmbDb8OFlY2f4X8CfgPuBFoNaNcDNrcB6GMjOzLF9ZmJlZ1oiZHGybbbaJyZMn93v/Z599li222GLgAhrm3B/rc3+8mvtkfcO1P5YuXbo6IrbN1RsxyWLy5MksWbKk3/t3dHTQ0tIycAENc+6P9bk/Xs19sr7h2h+SevvW/8s8DGVmZllOFmZmluVkYWZmWU4WZmaW5WRhZmZZThZmZpblZGFmZllOFmZmluVkYWZmWSPmG9wjTWtba83yxXMWD3Ikg2c0vmaz4cJXFmZmluVkYWZmWU4WZmaW5WRhZmZZThZmZpblZGFmZllOFmZmluVkYWZmWU4WZmaW5WRhZmZZThZmZpblZGFmZllOFmZmluVkYWZmWU4WZmaW5WRhZmZZThZmZpblZGFmZlmlJgtJ0yXdK6lT0gk1to+VdFHafoukyal8sqS/SLoj/ZxbZpxmZta70p7BLWkMcA6wP7AKuFXSooi4u6La0cATEbGLpFnA6cBhadt9EbFHWfGZmVn9yryy2BvojIgVEfEC0A7MqKozA2hLy5cA+0lSiTGZmVk/KCLKaViaCUyPiGPS+pHAtIiYV1HnrlRnVVq/D5gGNAHLgOXA08CXI+L6GseYC8wFmDhx4l7t7e39jrerq4umpqZ+7z/Qlq9ZXrN81wm7Dsrxh6I/hvo196bRzo9G4D5Z33Dtj9bW1qUR0ZyrV9ow1AZ6BNgxItZI2gv4L0m7R8TTlZUi4jzgPIDm5uZoaWnp9wE7OjrYkP0H2vy2+TXLFx+8eFCOPxT9MdSvuTeNdn40AvfJ+kZ6f5Q5DPUQsEPF+qRUVrOOpI2BccCaiHg+ItYARMRS4D5g6P+8NDMbpcpMFrcCUyTtLGlTYBawqKrOImBOWp4JXBsRIWnbdIMcSW8EpgArSozVzMx6UdowVESskzQPuAoYA5wfEcsknQIsiYhFwALgAkmdwFqKhAKwL3CKpL8CLwHHRsTasmI1M7PelXrPIiKuAK6oKjupYvk54JAa+10KXFpmbGZmVj9/g9vMzLKcLMzMLMvJwszMspwszMwsy8nCzMyynCzMzCzLycLMzLKcLMzMLMvJwszMspwszMwsy8nCzMyynCzMzCzLycLMzLKcLMzMLMvJwszMspwszMwsy8nCzMyynCzMzCzLycLMzLKcLMzMLMvJwszMspwszMwsy8nCzMyyNh7qAKzxtLa1MrtpNvPb5q9XvnjO4iGLp5ahisdsNPKVhZmZZTlZmJlZlpOFmZllOVmYmVlWqclC0nRJ90rqlHRCje1jJV2Utt8iaXLV9h0ldUk6vsw4zcysd6UlC0ljgHOAA4GpwGxJU6uqHQ08ERG7AGcCp1dtPwO4sqwYzcysPmVeWewNdEbEioh4AWgHZlTVmQG0peVLgP0kCUDSR4H7gWUlxmhmZnVQRJTTsDQTmB4Rx6T1I4FpETGvos5dqc6qtH4fMA14DrgG2B84HuiKiG/UOMZcYC7AxIkT92pvb+93vF1dXTQ1NfV7/4G2fM3ymuW7Tth1UI49fsx41r64dlCP3dNr7slg9EW3Rjs/GoH7ZH3DtT9aW1uXRkRzrl6jfinvZODMiOhKFxo1RcR5wHkAzc3N0dLS0u8DdnR0sCH7D7TqL8R1W3xw+V9Em982n9lNs1nYtXBQj93Ta+7JYPRFt0Y7PxqB+2R9I70/ykwWDwE7VKxPSmW16qyStDEwDlhDcXUxU9LXga2AlyQ9FxFnlxivmZn1oMxkcSswRdLOFElhFvDxqjqLgDnAzcBM4NooxsXe211B0skUw1BOFGZmQ6S0ZBER6yTNA64CxgDnR8QySacASyJiEbAAuEBSJ7CWIqGYmVmDKfWeRURcAVxRVXZSxfJzwCGZNk4uJTgzM6ubv8FtZmZZThZmZpblZGFmZllOFmZmluVkYWZmWU4WZmaW5WRhZmZZThZmZpblZGFmZll1JQtJP5X0IUlOLmZmo1C9b/7fpZgE8A+S/l3Sm0uMyczMGkxdySIifhERhwPvAFYCv5B0k6RPStqkzADNzGzo1T2sJGkCcBRwDHA7cBZF8rimlMjMzKxh1DXrrKTLgDcDFwAfjohH0qaLJC0pKzgzM2sM9U5R/v003fjLJI2NiOfreXarmZkNb/UOQ51ao+zmgQzEzMwaV69XFpJeD2wPbCZpT0Bp05bA5iXHZmZmDSI3DPUBipvak4AzKsqfAb5UUkxmZtZgek0WEdEGtEk6OCIuHaSYzMysweSGoY6IiAuByZKOq94eEWfU2M3MzEaY3DDUFul3U9mBmJlZ48oNQ30v/Z4/OOGYmVkjyg1Dfbu37RHxuYENx8zMGlFuGGrpoERhZmYNrZ5PQ5mZ2SiXG4b6VkR8QdLPgKjeHhEfKS0yMzNrGLlhqAvS72+UHYiZmTWu3DDU0vT7OkmbAm+huMK4NyJeGIT4zMysAdT7WNUPAfcB3wbOBjolHVjHftMl3SupU9IJNbaPlXRR2n6LpMmpfG9Jd6Sf30r6WF9elJmZDax6pyj/JtAaEZ0Akt4EXA5c2dMOksYA5wD7A6uAWyUtioi7K6odDTwREbtImgWcDhwG3AU0R8Q6SdsBv5X0s4hY18fXZ2ZmA6DeKcqf6U4UyQqKyQR7szfQGREr0pBVOzCjqs4MoPsTV5cA+0lSRPy5IjG8hho3183MbPAoouf3YUl/nxb3B3YCLqZ44z4E+GNEfKaXfWcC0yPimLR+JDAtIuZV1Lkr1VmV1u9LdVZLmgacn457ZERcVuMYc4G5ABMnTtyrvb297hderauri6amxpnVZPma5TXLd52w66Ace/yY8ax9ce2gHrun19yTweiLbo12fjQC98n6hmt/tLa2Lq3nIXa5YagPVyw/BvxdWn4c2KyfsdUlIm4Bdpe0G8XMt1dGxHNVdc4DzgNobm6OlpaWfh+vo6ODDdl/oM1vqz3DyuKDFw/KsWc3zWZh18JBPXZPr7kng9EX3Rrt/GgE7pP1jfT+yH0a6pMb0PZDwA4V65NSWa06qyRtDIwD1lTFcI+kLuCtgJ/3bWY2BOq6wS3pNRQ3o3enuIcAQER8qpfdbgWmSNqZIinMAj5eVWcRMIfiEa0zgWsjItI+D6Yb3DtRfGR3ZV2vyMzMBly9N7gvAF5P8eS86yiuEnq9wZ1uUM8DrgLuAS6OiGWSTpHU/c3vBcAESZ3AcUD3x2vfQ/EJqDuAy4DPRMTq+l+WmZkNpHo/OrtLRBwiaUZEtEn6CXB9bqeIuAK4oqrspIrl5yhullfvdwGvfHvczMyGWL1XFn9Nv5+U9FaKewuvKyckMzNrNPVeWZwnaWvgKxT3GZrSslVpbWvtU/3Fc/r2iZ6y2x+NeurTwei7oTy2WV/UlSwi4gdp8TrgjeWFY2ZmjajeuaEmSPqOpNskLZX0LUkTyg7OzMwaQ733LNqBPwEHU3zEdTVwUVlBmZlZY6n3nsV2EfHVivVTJR1WRkBmZtZ46r2yuFrSLEkbpZ9DKb4/YWZmo0DusarPUEwcKOALwIVp00ZAF3B8qdGZmVlDyM0N9drBCsTMzBpXvfcsSFN07JtWOyLi5+WEZGZmjabej87+O/B54O7083lJXyszMDMzaxz1Xll8ENgjIl4CkNQG3A6cWFZgZmbWOOr9NBTAVhXL4wY6EDMza1z1Xll8Dbhd0mKKT0btyyvTiZuZ2QiXTRaSBNwAvAt4Zyr+l4h4tMzAzMyscWSTRXpy3RUR8TaKGWfNzGyUqfeexW2S3pmvZmZmI1G99yymAUdIWgk8S3HfIiLib8oKzMzMGke9yeIDpUZhZmYNLTc31GuAY4FdgDuBBRGxbjACMzOzxpG7Z9EGNFMkigOBb5YekZmZNZzcMNTU9CkoJC0AflN+SGZm1mhyVxZ/7V7w8JOZ2eiVu7J4u6Sn07KAzdJ696ehtiw1OjMzawi551mMGaxAzMyscfVlIkEzMxulnCzMzCzLycLMzLJKTRaSpku6V1KnpFdNaS5prKSL0vZbJE1O5ftLWirpzvT7fWXGaWZmvSstWUgaA5xD8WW+qcBsSVOrqh0NPBERuwBnAqen8tXAh9N3POYAF5QVp5mZ5ZV5ZbE30BkRKyLiBaAdmFFVZwbFt8QBLgH2k6SIuD0iHk7lyyg+sju2xFjNzKwXiohyGpZmAtMj4pi0fiQwLSLmVdS5K9VZldbvS3VWV7VzbES8v8Yx5gJzASZOnLhXe3t7v+Pt6uqiqamp3/t3W75meZ/q7zph1wFpp6/t92b5muWMHzOetS+u3eC2+nrcvhioeHo6bmX7A3V+9OfYjaqsPhmuhmt/tLa2Lo2I5ly9hk4WknaneODSARFxX2/Ha25ujiVLlvQ73o6ODlpaWvq9f7fWttY+1V88Z/GAtNPX9nvT2tbK7KbZLOxauMFt9fW4fTFQ8fR03Mr2B+r86M+xG1VZfTJcDdf+kFRXsihzGOohYIeK9UmprGYdSRsD44A1aX0ScBnwiVyiMDOzcpWZLG4FpkjaWdKmwCxe/VjWRRQ3sAFmAtemx7huBVwOnBARN5YYo5mZ1aG0ZJEmHpwHXAXcA1wcEcsknSLpI6naAmCCpE7gOKD747XzKJ6hcZKkO9LP68qK1czMelfvk/L6JSKuAK6oKjupYvk54JAa+50KnFpmbGZmVj9/g9vMzLKcLMzMLMvJwszMspwszMwsy8nCzMyynCzMzCzLycLMzLKcLMzMLMvJwszMspwszMwsy8nCzMyynCzMzCzLycLMzLKcLMzMLMvJwszMspwszMwsq9SHHw0ny9csZ37b/FeVL56zuNTjtra1ltr+QB63p33K7qOelN13le3Pbpq93vkxVK+5JwP1b9OXPu3uk56O0Wjny1Aq+99nMPrUVxZmZpblZGFmZllOFmZmluVkYWZmWU4WZmaW5WRhZmZZThZmZpblZGFmZllOFmZmluVkYWZmWU4WZmaWVWqykDRd0r2SOiWdUGP7WEkXpe23SJqcyidIWiypS9LZZcZoZmZ5pSULSWOAc4ADganAbElTq6odDTwREbsAZwKnp/LngK8Ax5cVn5mZ1a/MK4u9gc6IWBERLwDtwIyqOjOAtrR8CbCfJEXEsxFxA0XSMDOzIaaIKKdhaSYwPSKOSetHAtMiYl5FnbtSnVVp/b5UZ3VaPwportyn6hhzgbkAEydO3Ku9vb3f8a5+cjVrX1z7qvJdJ+zap3aWr1ne7xjK0FP8uTjHjxlfsz/6coy+arS+q1TdH2W/5oE678o8f7v7pK/n2ED1XaPp6uqiqamp5rZG+3eu1NraujQimnP1hnWyqNTc3BxLlizpd7znXXoeC7sWvqq8zOcBDIa+Pmug2+ym2TX7oy/H6KtG67tK1f1R9msequcc9PV5Fgu7Fvp5FklHRwctLS01tzXav3MlSXUlizKHoR4CdqhYn5TKataRtDEwDlhTYkxmZtYPZSaLW4EpknaWtCkwC1hUVWcRMCctzwSujbIudczMrN9Ke6xqRKyTNA+4ChgDnB8RyySdAiyJiEXAAuACSZ3AWoqEAoCklcCWwKaSPgocEBF3lxWvmZn1rNRncEfEFcAVVWUnVSw/BxzSw76Ty4zNzMzq529wm5lZlpOFmZllOVmYmVmWk4WZmWU5WZiZWZaThZmZZTlZmJlZlpOFmZllOVmYmVmWk4WZmWU5WZiZWZaThZmZZTlZmJlZlpOFmZllOVmYmVmWk4WZmWU5WZiZWZaThZmZZTlZmJlZlpOFmZllOVmYmVmWk4WZmWU5WZiZWZaThZmZZTlZmJlZlpOFmZllOVmYmVmWk4WZmWWVmiwkTZd0r6ROSSfU2D5W0kVp+y2SJldsOzGV3yvpA2XGaWZmvSstWUgaA5wDHAhMBWZLmlpV7WjgiYjYBTgTOD3tOxWYBewOTAe+m9ozM7MhUOaVxd5AZ0SsiIgXgHZgRlWdGUBbWr4E2E+SUnl7RDwfEfcDnak9MzMbAoqIchqWZgLTI+KYtH4kMC0i5lXUuSvVWZXW7wOmAScDv46IC1P5AuDKiLik6hhzgblp9c3AvRsQ8jbA6g3Yf6Rxf6zP/fFq7pP1Ddf+2Ckits1V2ngwIilLRJwHnDcQbUlaEhHNA9HWSOD+WJ/749XcJ+sb6f1R5jDUQ8AOFeuTUlnNOpI2BsYBa+rc18zMBkmZyeJWYIqknSVtSnHDelFVnUXAnLQ8E7g2inGxRcCs9GmpnYEpwG9KjNXMzHpR2jBURKyTNA+4ChgDnB8RyySdAiyJiEXAAuACSZ3AWoqEQqp3MXA3sA74h4h4saxYkwEZzhpB3B/rc3+8mvtkfSO6P0q7wW1mZiOHv8FtZmZZThZmZpY16pNFbkqSkU7SDpIWS7pb0jJJn0/l4yVdI+kP6ffWQx3rYJM0RtLtkn6e1ndO09J0pmlqNh3qGAeLpK0kXSLp95LukfTu0X6OSPrH9H/mLkkLJb1mJJ8jozpZ1DklyUi3DviniJgKvAv4h9QHJwC/jIgpwC/T+mjzeeCeivXTgTPT9DRPUExXM1qcBfz/iHgL8HaKfhm154ik7YHPAc0R8VaKD/HMYgSfI6M6WVDflCQjWkQ8EhG3peVnKN4Etmf9qVjagI8OTYRDQ9Ik4EPAD9K6gPdRTEsDo6hPJI0D9qX49CIR8UJEPMkoP0coPk26WfqO2ObAI4zgc2S0J4vtgQcr1lelslEpzfq7J3ALMDEiHkmbHgUmDlFYQ+VbwD8DL6X1CcCTEbEurY+mc2Vn4HHgh2lY7geStmAUnyMR8RDwDeCPFEniKWApI/gcGe3JwhJJTcClwBci4unKbemLkqPmM9aSDgL+FBFLhzqWBrEx8A7gPyJiT+BZqoacRuE5sjXFldXOwBuALShmyB6xRnuy8LQigKRNKBLFjyPip6n4MUnbpe3bAX8aqviGwN8CH5G0kmJo8n0UY/ZbpSEHGF3nyipgVUTcktYvoUgeo/kceT9wf0Q8HhF/BX5Kcd6M2HNktCeLeqYkGdHSWPwC4J6IOKNiU+VULHOA/x7s2IZKRJwYEZMiYjLFOXFtRBwOLKaYlgZGUZ9ExKPAg5LenIr2o5hdYdSeIxTDT++StHn6P9TdJyP2HBn13+CW9EGK8enuKUlOG+KQBpWk9wDXA3fyyvj8lyjuW1wM7Ag8ABwaEWuHJMghJKkFOD4iDpL0RoorjfHA7cAREfH8UMY3WCTtQXGzf1NgBfBJij82R+05Imk+cBjFJwpvB46huEcxIs+RUZ8szMwsb7QPQ5mZWR2cLMzMLMvJwszMspwszMwsy8nCzMyynCxs2JF0pqQvVKxfJekHFevflHRcpo2b6jjOSknb1ChvkbRPX+PeEJKOknR2Wj5W0icqyt/Qj/ZqvjaznjhZ2HB0I7APgKSNgG2A3Su27wP0mgwiYkPe7Fu6jz8UIuLciPjPtHoUxXQTZqVysrDh6Cbg3Wl5d+Au4BlJW0saC+wG3AYg6YuSbpX0u/QlKlJ5V/q9kaTvpuc0XCPpCkkzK471WUm3SbpT0lvSZIvHAv8o6Q5J760MTNLekm5OE+7d1P2t53QF8F/pGCslzZN0XKr3a0njU70OSWeltu+StHf1i5d0sqTjU5zNwI9T/c0qrxgkNUvqSMsTJF2dnr/wA0AV7R0h6Tepje+lqfvN1uNkYcNORDwMrJO0I8Vf+DdTfOP83RRvnndGxAuSDgCmUExFvwewl6R9q5r7e2AyxfNMjuSVJNRtdUS8A/gPim9yrwTOpXhmwR4RcX1V/d8D700T7p0E/FvFtrem470TOA34c6p3M/CJinqbR8QewGeA83vph0uAJcDhKZa/9FQX+FfghojYHbiM4lvXSNqN4lvIf5uO+SJweC/t2Ci1cb6KWUO6iSJR7AOcQTHNwj4UU0XfmOockH5uT+tNFMnjVxXtvAf4fxHxEvCopMVVx+meWHEpxRt9zjigTdIUillYN6nYtjg9M+QZSU8BP0vldwJ/U1FvIUBE/ErSlpK2quO4OfuS4o+IyyU9kcr3A/YCbi2mOGIzRteEgFYnJwsbrrrvW7yNYhjqQeCfgKeBH6Y6Ar4WEd/bgON0z+vzIvX9f/kqRVL4WBqy6qjRFhTzcD1fsVzZdvUcPH2Zk2cdr4wYvKaO+gLaIuLEPhzDRiEPQ9lwdRNwELA2Il5ME9htRTGM1H1z+yrgU+lZHUjaXtLrqtq5ETg43buYSHHzOucZ4LU9bBvHK9NSH1Xna6l2GLw8yeNTEfFUH2JZSXGlAHBwRfmvgI+ndg8Eup+X/UtgZne/qHiu9k79jNtGMCcLG67upPgU1K+ryp6KiNUAEXE18BPgZkl3UjyHofpN/lKK5zXcDVxIcWO8tzdnKIaPPlbrBjfwdeBrkm6n/1fuz6X9zyX/DOcfAed23+AG5gNnSVpCcTXUbT6wr6RlFMNRfwSIiLuBLwNXS/odcA2wXT/jthHMs87aqCepKSK6JE0AfkNxs/fRIYorArgfAAAARElEQVSlg+JG+pKhOL5ZT3zPwgx+nm4ibwp8dagShVkj85WFmZll+Z6FmZllOVmYmVmWk4WZmWU5WZiZWZaThZmZZf0PHG7XpzOWxHgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAHg1JREFUeJzt3XuYHGWd9vHvTTgIDKcQjUqQoAQ1oHIYAU+YWVGDB1iXqImCoGLWfTeKi7oLuiJBWTyirLgLUYSsYCKCvosYXkDNgMjBJKBiQCDGyCkcQjgNKhj4vX/U00XR0z3dM9M13Z25P9c113RVP/3U3ZVJ/7qe6npaEYGZmRnAJu0OYGZmncNFwczMci4KZmaWc1EwM7Oci4KZmeVcFMzMLOeiYMMm6QxJn2my7TmSPl92plaQdKKkc9PtF0gakDShRX1PlRSSNm1BXzMk3VlYXilpxmj7TX29V9JlheWQtFsr+k79DUh6Yav6s9ZzURgHJB0v6ZKqdbfVWTe7UX8R8eGI+FyLsrX0RadVIuL2iOiJiCcBJPVLOrrduWqJiD0ion+oNs0WpYg4LyLe1IpctfZZ2qerW9G/lcNFYXy4Enh15V2vpOcBmwF7V63bLbW1cagVRzHW/VwUxodlZEVgr7T8OmApcEvVuj9ExN0Akl4i6XJJ6yXdIuldlc6qh4Qk/auktZLulnR0jXf/O0j6iaRHJV0n6UXpcZUC9Js0rPDu6uCSXiTp55IekLRO0nmSti/cv0bSJyX9VtJjks6SNFnSJWl7P5W0Q2pbebc8N2VdK+kTtXZY8Z21pJPT/jk95Ty91jvv4jtjSRMkfSVlXg28tar/7VLWtZLukvT5ekNVkrZM+/xBSTcBr6y6f42kg9Lt/SQtl/SIpHslnZqaVfb1Q+k5vErSUZJ+Kelrkh4ATkzrrqqK8BZJq9Nz+bKkTdK28uG2ZvZZapP/baR98D+S7pf0J0n/Xuj7KElXpX34oKQ/Sjq41v6x1nJRGAci4gngOuDAtOpA4BfAVVXrrgSQtDVwOfA94DnAbOC/JE2v7lvSTOBY4CCyI40ZNSLMBuYDOwCrgJNTrsq2X5GGFb5f47ECTgGeD7wU2Bk4sarNYcAbgd2BtwOXAJ8Cnk32N/7RqvZ9wDTgTcC/VV5Q64mIT5Ptr3kp57yh2icfAt4G7A30ArOq7j8H2EC2z/ZOWeoNT30WeFH6eTNw5BDbPQ04LSK2Te3PT+sr+3r79ByuScv7A6uByaR/lxrekZ7DPsChwAeG2D7Q9D77BrAd8ELg9cD7gPcX7t+f7I3LJOBLwFmS1GjbNjouCuPHFTz9wvA6sv+wv6had0W6/TZgTUScHREbIuIG4ELgnTX6fRdwdkSsjIg/M/gFG+BHEfGriNgAnMfTRycNRcSqiLg8Ih6PiPuBU8leQIq+ERH3RsRd6TldFxE3RMRfgR+RvegWzY+IxyLiRuBsYE6zeYbhXcDXI+KOiFhPVtgAkDQZeAvwsZTjPuBrZMWzXl8nR8T6iLgD+M8htvs3YDdJkyJiICKubZDz7oj4Rvp3/kudNl9M274d+Dot2F/pqGg2cHxEPBoRa4CvAkcUmv0pIr6VzussBJ5HVrysRC4K48eVwGslTQSeHRG3AVeTnWuYCOzJ00MMuwD7S3qo8gO8F3hujX6fD9xRWL6jRpt7Crf/DPQ0GzoNBS1OQyyPAOeSvXMsurdw+y81lqu3V8z4J7Ln0GrV++VPhdu7kA3nrS3s3zPJjsqG21e1D5IdMf1e0jJJb2uQs9a/11BtWrW/JpHtg+Jz+ROwU2E5/7tJbzhgGH87NjIuCuPHNWSH6h8CfgkQEY8Ad6d1d0fEH1PbO4ArImL7wk9PRPxTjX7XAlMKyzu3OPd/AAG8LA2JHE42pDQaxYwvINsHjVRPJ/xY+r1VYV2xaK6tsZ2KO4DHgUmF/bttROxRZ9tD9fXMkBG3RcQcsgLzReCCNBxYbzrkZqZJrre/HqP+82/U9zqyo5pdqvq+q4k8ViIXhXEiDQ0sJxv//0XhrqvSuuKnji4Gdpd0hKTN0s8rJb20RtfnA++X9FJJWwFNXb9QcC/ZmHI92wADwMOSdgI+Ocz+a/mMpK0k7UE2hl3rXEa1Z+RMQ1l3AYenk8ofIBvDrzgf+KikKelE93GFx64FLgO+KmlbSZsoO6FePSxW7Ot4STtImgJ8pF5ISYdLenZEPAU8lFY/Bdyffo/kGoFPpm3vDBzD0/vr18CByq7p2A44vupxdf9t05DQ+cDJkraRtAvZ3+G5tdrb2HFRGF+uIHsHWfx0yS/SurwoRMSjZCc+Z5O9K7yH7F3nFtUdRsQlZGPcS8lOIlfGsB9vMtOJwMI0jPKuGvfPJzvB+TDwE+CHTfY7lCvIsv4M+EpEXNagPWQncGelT8JUxvQ/RFakHgD2IBuOq/gWcCnwG+D6GrnfB2wO3AQ8CFxANmZey3yyoZU/khWT7w6RcyawUtJAyjw7Iv6Shl9OBn6Z9vUBjZ9y7n+BFWRF4CfAWQARcTlZgfhtuv/iqsfV2mdFHyE72lhN9jf5PeA7w8hlJZC/ZMdaKR1N/A7YIp1Y7hiSppK9sG7WadnMOoWPFGzUJL1D0hZpmOSLwI/9omvWnVwUrBX+EbgP+APwJFDrhLSZdQEPH5mZWc5HCmZmluu6CbAmTZoUU6dOHdFjH3vsMbbeeuvWBipRN+XtpqzQXXm7KSt0V95uygqjy7tixYp1EfHshg0joqt+9t133xippUuXjvix7dBNebspa0R35e2mrBHdlbebskaMLi+wPJp4jfXwkZmZ5VwUzMws56JgZmY5FwUzM8u5KJiZWc5FwczMci4KZmaWc1EwM7Oci4KZmeW6bpqL0bj1gVuZv3D+oPVLj1zahjRmZp3HRwpmZpZzUTAzs5yLgpmZ5VwUzMws56JgZmY5FwUzM8u5KJiZWc5FwczMci4KZmaWc1EwM7Oci4KZmeVcFMzMLOeiYGZmORcFMzPLuSiYmVnORcHMzHIuCmZmlnNRMDOznIuCmZnlXBTMzCznomBmZjkXBTMzy7komJlZzkXBzMxyLgpmZpZzUTAzs1ypRUHSTEm3SFol6bgh2h0mKST1lpnHzMyGVlpRkDQB+CZwMDAdmCNpeo122wDHANeVlcXMzJpT5pHCfsCqiFgdEU8Ai4FDa7T7HPBF4K8lZjEzsyYoIsrpWJoFzIyIo9PyEcD+ETGv0GYf4NMRcZikfuATEbG8Rl9zgbkAkydP3nfx4sUjyrTuoXWsf3L9oPW777j7iPor28DAAD09Pe2O0ZRuygrdlbebskJ35e2mrDC6vH19fSsiouEQ/aYj6r0FJG0CnAoc1ahtRCwAFgD09vbGjBkzRrTNBRcuYNHAokHrlx62dET9la2/v5+RPtex1k1ZobvydlNW6K683ZQVxiZvmcNHdwE7F5anpHUV2wB7Av2S1gAHABf5ZLOZWfuUWRSWAdMk7Sppc2A2cFHlzoh4OCImRcTUiJgKXAscUmv4yMzMxkZpRSEiNgDzgEuBm4HzI2KlpJMkHVLWds3MbORKPacQEUuAJVXrTqjTdkaZWczMrDFf0WxmZjkXBTMzy7komJlZrm3XKdjQ+hb2MadnDvMXzn/G+qVHduY1FWa2cfCRgpmZ5VwUzMws56JgZmY5FwUzM8u5KJiZWc5FwczMci4KZmaWc1EwM7Oci4KZmeV8RfNGrm9hX831nXhldDdlNdtY+UjBzMxyLgpmZpZzUTAzs5yLgpmZ5VwUzMws56JgZmY5FwUzM8u5KJiZWc5FwczMcr6ieYR89e345X9725j5SMHMzHIuCmZmlnNRMDOznIuCmZnlXBTMzCznomBmZjkXBTMzy7komJlZzkXBzMxyLgpmZpZzUTAzs1ypRUHSTEm3SFol6bga939Y0o2Sfi3pKknTy8xjZmZDK60oSJoAfBM4GJgOzKnxov+9iHhZROwFfAk4taw8ZmbWWJlHCvsBqyJidUQ8ASwGDi02iIhHCotbA1FiHjMza0ARjV+HJf0QOAu4JCKeaqpjaRYwMyKOTstHAPtHxLyqdv8MHAtsDvxdRNxWo6+5wFyAyZMn77t48eJmIgyy7qF1rH9y/aD1u++4+7D7uvWBW2uuH0lf9fqfOGHioLzD7b/snBUDAwP09PSMqo+xygqjyzuWOaE1+3YsdVPebsoKo8vb19e3IiJ6G7VrtigcBLwfOAD4AXB2RNzS4DFNFYVC+/cAb46II4fqt7e3N5YvX94wcy0LLlzAooFFg9aPZB78sufU71vYx5yeOYPyDrf/sZr7v7+/nxkzZoyqj7H8noLR5B3r71Noxb4dS92Ut5uywujySmqqKDQ1fBQRP42I9wL7AGuAn0q6WtL7JW1W52F3ATsXlqekdfUsBv6+mTxmZlaOps8pSNoROAo4GrgBOI2sSFxe5yHLgGmSdpW0OTAbuKiqz2mFxbcCg4aOzMxs7DT1dZySfgS8GPgu8PaIWJvu+r6kmmM5EbFB0jzgUmAC8J2IWCnpJGB5RFwEzEtDU38DHgSGHDoyM7NyNfsdzd+KiCXFFZK2iIjHhxqjSo9ZUrXuhMLtY4YT1szMytXs8NHna6y7ppVBzMys/YY8UpD0XGAnYEtJewNKd20LbFVyNjMzG2ONho/eTHZyeQrPvNr4UeBTJWUyM7M2GbIoRMRCYKGkwyLiwjHKZGZmbdJo+OjwiDgXmCrp2Or7I8JzFZmZbUQaDR9tnX53z3XgZmY2Yo2Gj85Mv+ePTRwzM2unRsNH/znU/RHx0dbGMTOzdmo0fLRiTFJYV6tM3jd/4TMPKMuaIG6kihPZFfN2Wk6zdmrm00dmZjZONBo++npEfEzSj6nxBTgRcUhpyczMbMw1Gj76bvr9lbKDmJlZ+zUaPlqRfl+Rpr9+CdkRwy3pKzbNzGwj0uzU2W8FzgD+QDb/0a6S/jEiLikznJmZja1mp87+KtAXEasAJL0I+AngomBmthFpdursRysFIVlNNimemZltRBp9+ugf0s3lkpYA55OdU3gn2ddtmpnZRqTR8NHbC7fvBV6fbt8PbFlKIjMza5tGnz56/1gFMTOz9mv200fPAj4I7AE8q7I+Ij5QUi4zM2uDZk80fxd4Ltk3sV1B9k1sPtFsZraRabYo7BYRnwEeS/MhvRXYv7xYZmbWDs0Whb+l3w9J2hPYDnhOOZHMzKxdmr14bYGkHYDPABeRfRPbZ0pLZWZmbdFUUYiIb6ebVwAvLC+OmZm1U1PDR5J2lPQNSddLWiHp65J2LDucmZmNrWbPKSwG7gMOA2YB64DvlxXKzMzao9lzCs+LiM8Vlj8v6d1lBDIzs/Zp9kjhMkmzJW2Sft4FXFpmMDMzG3uNJsR7lGwCPAEfA85Nd20CDACfKDWdmZmNqUZzH20zVkHMzKz9mj2ngKRDgAPTYn9EXFxOJDMza5dmP5L6BeAY4Kb0c4ykU8oMZmZmY6/ZI4W3AHtFxFMAkhYCNwDHlxXMzMzGXrOfPgLYvnB7u1YHMTOz9mu2KJwC3CDpnHSUsAI4udGDJM2UdIukVZKOq3H/sZJukvRbST+TtMvw4puZWSs1HD6SJOAq4ADglWn1v0XEPQ0eNwH4JvBG4E5gmaSLIuKmQrMbgN6I+LOkfwK+BPiiODOzNmlYFCIiJC2JiJeRzZDarP2AVRGxGkDSYuBQshPVlb6XFtpfCxw+jP7NzKzFFBGNG2VDRqdHxLKmO5ZmATMj4ui0fASwf0TMq9P+dOCeiPh8jfvmAnMBJk+evO/ixYubjfEM6x5ax/on1w9av/uOuw+7r1sfuLXm+pH0Va//iRMmDso73P7LzlnZRjdkLfZfzNtpOasNDAzQ09PT0j7LfA5l5C1LN2WF0eXt6+tbERG9jdo1WxR+D0wD1gCPkV3hHBHx8iEe03RRkHQ4MA94fUQ8PlSW3t7eWL58ecPMtSy4cAGLBhYNWr/0yKU1Wg+tb2FfzfUj6ate/3N65gzKO9z+y85Z2UY3ZC32X8zbaTmr9ff3M2PGjJb2WeZzKCNvWbopK4wur6SmikKzH0l98wgy3AXsXFiektY9g6SDgE/TREEwM7NyNZr76FnAh4HdgBuBsyJiQ5N9LwOmSdqVrBjMBt5T1f/ewJlkRxT3DTO7mZm1WKOPpC4EeskKwsHAV5vtOBWPeWSzqd4MnB8RKyWdlKbMAPgy2Vd7/kDSryUN50S2mZm1WKPho+npU0dIOgv41XA6j4glwJKqdScUbh80nP7MzKxcjY4U/la5MYxhIzMz61KNjhReIemRdFvAlmm58umjbUtNZ2ZmY6rR9ylMGKsgZmbWfsOZEM/MzDZyLgpmZpZzUTAzs5yLgpmZ5VwUzMws56JgZmY5FwUzM8u5KJiZWc5FwczMci4KZmaWc1EwM7Ncs9+8ZmbWFmP99afjnY8UzMws56JgZmY5FwUzM8u5KJiZWc5FwczMci4KZmaWc1EwM7Oci4KZmeVcFMzMLOcrms06UPEq3jk9c5i/cD7QeVfx1rraeE7PHGYwY+zDWEv4SMHMzHIuCmZmlnNRMDOznIuCmZnlXBTMzCznomBmZjkXBTMzy7komJlZzhevmdm44AvtmuMjBTMzy5VaFCTNlHSLpFWSjqtx/4GSrpe0QdKsMrOYmVljpRUFSROAbwIHA9OBOZKmVzW7HTgK+F5ZOczMrHllnlPYD1gVEasBJC0GDgVuqjSIiDXpvqdKzGFmZk0qc/hoJ+COwvKdaZ2ZmXUoRUQ5HWfnCGZGxNFp+Qhg/4iYV6PtOcDFEXFBnb7mAnMBJk+evO/ixYtHlGndQ+tY/+T6Qet333H3Yfd16wO31lw/kr7q9T9xwsRBeYfbf9k5K9vohqzF/ot5Oy1n9TZGk7WZ/otasS8mTpjIpO0njShXs9uAzsxatoGBAXp6ekb02L6+vhUR0duoXZlF4VXAiRHx5rR8PEBEnFKj7TkMURSKent7Y/ny5SPKtODCBSwaWDRo/UjmqK/18baR9lWv/zk9cwblHW7/ZeesbKMbslZ/R0Elb6flrN7GaLI2039RK/bFnJ45zD1s7ohyNbsN6MysZevv72fGjBkjeqykpopCmcNHy4BpknaVtDkwG7ioxO2ZmdkolVYUImIDMA+4FLgZOD8iVko6SdIhAJJeKelO4J3AmZJWlpXHzMwaK/WK5ohYAiypWndC4fYyYEqZGczMrHm+otnMzHIuCmZmlnNRMDOznIuCmZnlXBTMzCznomBmZjkXBTMzy7komJlZzkXBzMxyLgpmZpZzUTAzs5yLgpmZ5VwUzMws56JgZmY5FwUzM8u5KJiZWc5FwczMci4KZmaWc1EwM7Ncqd/RbGZmw9e3sK/m+s/u8tnSt+0jBTMzy7komJlZzkXBzMxyLgpmZpZzUTAzs5yLgpmZ5VwUzMws56JgZmY5FwUzM8v5imYzsxaodxXy0iOXjnGS0fGRgpmZ5VwUzMws56JgZmY5FwUzM8u5KJiZWc5FwczMcqUWBUkzJd0iaZWk42rcv4Wk76f7r5M0tcw8ZmY2tNKKgqQJwDeBg4HpwBxJ06uafRB4MCJ2A74GfLGsPGZm1liZRwr7AasiYnVEPAEsBg6tanMosDDdvgB4gySVmMnMzIagiCinY2kWMDMijk7LRwD7R8S8QpvfpTZ3puU/pDbrqvqaC8xNiy8GbhlhrEnAuoatOkc35e2mrNBdebspK3RX3m7KCqPLu0tEPLtRo66Y5iIiFgALRtuPpOUR0duCSGOim/J2U1borrzdlBW6K283ZYWxyVvm8NFdwM6F5SlpXc02kjYFtgMeKDGTmZkNocyisAyYJmlXSZsDs4GLqtpcBByZbs8Cfh5ljWeZmVlDpQ0fRcQGSfOAS4EJwHciYqWkk4DlEXERcBbwXUmrgPVkhaNMox6CGmPdlLebskJ35e2mrNBdebspK4xB3tJONJuZWffxFc1mZpZzUTAzs9y4KQqNptzoFJJ2lrRU0k2SVko6pt2ZmiFpgqQbJF3c7ixDkbS9pAsk/V7SzZJe1e5MQ5H0L+nv4HeSFkl6VrszFUn6jqT70jVHlXUTJV0u6bb0e4d2Zqyok/XL6W/ht5J+JGn7dmasqJW1cN/HJYWkSWVse1wUhSan3OgUG4CPR8R04ADgnzs4a9ExwM3tDtGE04D/FxEvAV5BB2eWtBPwUaA3IvYk+8BG2R/GGK5zgJlV644DfhYR04CfpeVOcA6Ds14O7BkRLwduBY4f61B1nMPgrEjaGXgTcHtZGx4XRYHmptzoCBGxNiKuT7cfJXvR2qm9qYYmaQrwVuDb7c4yFEnbAQeSfeqNiHgiIh5qb6qGNgW2TNfxbAXc3eY8zxARV5J9crCoOH3NQuDvxzRUHbWyRsRlEbEhLV5Ldj1V29XZr5DNEfevQGmfEBovRWEn4I7C8p10+AstQJo1dm/guvYmaejrZH+oT7U7SAO7AvcDZ6ehrm9L2rrdoeqJiLuAr5C9K1wLPBwRl7U3VVMmR8TadPseYHI7wwzDB4BL2h2iHkmHAndFxG/K3M54KQpdR1IPcCHwsYh4pN156pH0NuC+iFjR7ixN2BTYB/jviNgbeIzOGdoYJI3FH0pWzJ4PbC3p8PamGp50MWrHf+5d0qfJhm7Pa3eWWiRtBXwKOKHsbY2XotDMlBsdQ9JmZAXhvIj4YbvzNPAa4BBJa8iG5f5O0rntjVTXncCdEVE58rqArEh0qoOAP0bE/RHxN+CHwKvbnKkZ90p6HkD6fV+b8wxJ0lHA24D3dvCMCi8ie3Pwm/R/bQpwvaTntnpD46UoNDPlRkdIU4efBdwcEae2O08jEXF8REyJiKlk+/XnEdGR72Yj4h7gDkkvTqveANzUxkiN3A4cIGmr9HfxBjr4xHhBcfqaI4H/bWOWIUmaSTb0eUhE/LndeeqJiBsj4jkRMTX9X7sT2Cf9TbfUuCgK6URSZcqNm4HzI2Jle1PV9RrgCLJ33L9OP29pd6iNyEeA8yT9FtgL+I8256krHdFcAFwP3Ej2/7WjpmWQtAi4BnixpDslfRD4AvBGSbeRHe18oZ0ZK+pkPR3YBrg8/V87o60hkzpZx2bbnXu0ZGZmY21cHCmYmVlzXBTMzCznomBmZjkXBTMzy7komJlZzkXBOpakr0n6WGH5UknfLix/VdKxDfq4uontrKk146SkGZLG9GIxSUdJOj3d/rCk9xXWP38E/dV8bmb1uChYJ/sl6QpeSZsAk4A9Cve/GhjyRT8iRvOiPoM2XkEcEWdExP+kxaPIprowK5WLgnWyq4HK9x3sAfwOeFTSDpK2AF5KdmEXkj4paVmaF39+pQNJA+n3JpL+K82df7mkJZJmFbb1EUnXS7pR0kvSZIQfBv4lXdT0umIwSftJuiZNrHd15Srp9I7+/6ZtrJE0T9Kxqd21kiamdv2STkt9/07SftVPXtKJkj6RcvaSXXT3a0lbFo8AJPVK6k+3d5R0mbLvYPg2oEJ/h0v6VerjzDSlvNkzuChYx4qIu4ENkl5A9o79GrIZY19F9iJ5Y0Q8IelNwDSyKdL3AvaVdGBVd/8ATCX7Po0jeLrYVKyLiH2A/wY+ERFrgDOAr0XEXhHxi6r2vwdelybWO4FnXhm9Z9reK4GTgT+ndtcA7yu02yoi9gL+D/CdIfbDBcBysrl59oqIv9RrC3wWuCoi9gB+BLwAQNJLgXcDr0nbfBJ47xD92Di1absDmDVwNVlBeDVwKtmU568GHiYbXoLsS0feBNyQlnvIisSVhX5eC/wgIp4C7pG0tGo7lYkHV5C9oDeyHbBQ0jSyWUA3K9y3NH0XxqOSHgZ+nNbfCLy80G4RZHPnS9pWrfnWrwNJ+SPiJ5IeTOvfAOwLLMumUWJLOnyiOmsPFwXrdJXzCi8jGz66A/g48Ahwdmoj4JSIOHMU23k8/X6S5v5ffI7sxf8daaipv0ZfkH3HxOOF28W+q+eYGc6cMxt4+ki/ma/oFLAwIjrlm8WsQ3n4yDrd1WTTGq+PiCcjYj2wPdnwT+Uk86XAB5R9BwWSdpL0nKp+fgkcls4tTCY7idzIo2STpdWyHU9Pv35Uk8+l2rsBJL2W7At0Hh5GljVk7/wBDiusvxJ4T+r3YKDy/cg/A2ZV9ouy71HeZYS5bSPmomCd7kayTx1dW7Xu4YhYB9lXKgLfA66RdCPZzKLVL+YXkk03fBNwLtkJ6qFehCEb9nlHrRPNwJeAUyTdwMiPuP+aHn8G0GgWzHOAMyonmoH5wGmSlpMd3VTMBw6UtJJsGOl2gIi4Cfh34LI0Q+zlwPNGmNs2Yp4l1cYNST0RMSBpR+BXZCddWz4ffZNZ+slOaC9vx/bN6vE5BRtPLk4nczcHPteugmDWyXykYGZmOZ9TMDOznIuCmZnlXBTMzCznomBmZjkXBTMzy/1/GJGITAYBoHUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "approx size 27\n",
      "\n",
      "Test set: Accuracy: 9736/10000 (97.36%)\n",
      "\n"
     ]
    }
   ],
   "source": [
    "prec=(3, 7, 5) # (3, 7, 5)\n",
    "data_prec, proj_prec, diag_prec = prec \n",
    "model, data_prec = test_compression(prec=prec, rm_outlier_frac=99.9, show_distrib=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So, as you observe, we keep a 98% accuracy with weight and data inputs integers between 0 and 16 (so on 4 bits)!.\n",
    "Experiments show that the total output is around 27 bits long, so we'll construct a discrete log algorithm which can work for output up to 30 bits long (so < 10^9)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's save this model now"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "saving_path = '/Users/tryffel/code/reading-in-the-dark/mnist/objects/ml_models/quad_conv.pt'\n",
    "torch.save(model.state_dict(), saving_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Collateral efficiency\n",
    "\n",
    "We do a quick check to see if the collateral accuracy has changed, for example on the K-NN. You can observe that it hasn't."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_input_onehot_labels(dataset, label=\"font\", one_hot=True):\n",
    "    data_input = dataset.tensors[0]\n",
    "    label_idx = {'char': 0, 'font': 1}[label]\n",
    "    label_size = {'char': N_CHARS, 'font': N_FONTS}[label]\n",
    "    labels = dataset.tensors[1][:, label_idx].view(-1, 1)\n",
    "    \n",
    "    data_label_onehot = torch.zeros(len(labels), label_size)\n",
    "    data_label_onehot.scatter_(1, labels, 1)\n",
    "    \n",
    "    return data_input, labels, data_label_onehot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training set 60000 items\n",
      "Testing set  10000 items\n"
     ]
    }
   ],
   "source": [
    "train_dataset, test_dataset = collateral.data.get_datasets(None, None)\n",
    "train_input, train_label, train_label_one_hot = get_input_onehot_labels(train_dataset, label=\"font\")\n",
    "test_input, test_label, test_label_one_hot = get_input_onehot_labels(test_dataset, label=\"font\")\n",
    "\n",
    "data_prec, proj_prec, diag_prec = prec \n",
    "train_input = fix_precision(train_input, data_prec, parameter=False)\n",
    "test_input = fix_precision(test_input, data_prec, parameter=False)\n",
    "\n",
    "train_input = model.quad(train_input)\n",
    "test_input = model.quad(test_input)\n",
    "\n",
    "train_input = float_precision(train_input, sum(prec))\n",
    "test_input = float_precision(test_input, sum(prec))\n",
    "\n",
    "train_input = train_input.detach().numpy()\n",
    "test_input = test_input.detach().numpy()\n",
    "\n",
    "train_label = train_label.reshape(-1)\n",
    "test_label = test_label.reshape(-1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "ALL = train_input.shape[0]\n",
    "CPOWER = 'LOW'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "def evaluate_sklearn(reg, one_hot=True, limit=int(10e10)):\n",
    "    train_labels = {True: train_label_one_hot, False: train_label}[one_hot]\n",
    "    reg.fit(train_input[:limit], train_labels[:limit].detach().numpy()) \n",
    "    output = reg.predict(test_input)\n",
    "    if one_hot:\n",
    "        pred = torch.tensor(output).argmax(1, keepdim=True)\n",
    "    else:\n",
    "        if isinstance(output, list):\n",
    "            pred = torch.tensor(list(map(round, output))).long().view(-1, 1)\n",
    "        else:\n",
    "            pred = torch.tensor(np.round(output)).long().view(-1, 1)\n",
    "    y = test_label.view_as(pred)\n",
    "    acc = pred.eq(y).sum().item() / len(pred)\n",
    "    return acc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.6345\n"
     ]
    }
   ],
   "source": [
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "clf = KNeighborsClassifier(n_neighbors=7)\n",
    "acc = evaluate_sklearn(clf, one_hot=False)\n",
    "print(acc)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion\n",
    "\n",
    "Now that we know how to efficiently convert our model into integers, let's sync with the work of the paper to use their implementation of Functional Encryption, available [here](https://github.com/edufoursans/reading-in-the-dark)!\n",
    "\n",
    "Note that because we kept our output really low (<200M), we can reimplement the BabyStepGiantStep algorithm of the repo with small giant steps (=1000) and store all of them in a simple pickle file. Therefore, we show in practice that the discrete log computation drops from 11s to 20ms!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Export the model for FE** To get the model to the right format, you should convert the model paramaters (proj1 & diag1 weights) to lists (using model.proj1.weight.t().tolist()) and then follow the intructions in \"Supplementary Appendix B - Export model to Reading In the Dark.ipynb\". "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We do a similar approach with a model robust model in part 9 bis, and also assess how the CNN against which the resistance was built behave when the quadratic private network is converted to integers. Go check it out!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}