diff --git a/your-code/main (1).ipynb b/your-code/main (1).ipynb
new file mode 100644
index 0000000..df4c13e
--- /dev/null
+++ b/your-code/main (1).ipynb
@@ -0,0 +1,1369 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.6.5"
+ },
+ "colab": {
+ "name": "main.ipynb",
+ "provenance": []
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "04XdaynY2UQX"
+ },
+ "source": [
+ "#### 1. Import pandas library"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "metadata": {
+ "id": "qdvB-RE12UQc"
+ },
+ "source": [
+ "import pandas as pd"
+ ],
+ "execution_count": 1,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "eN5H2eba2UQd"
+ },
+ "source": [
+ "#### 2. Import pymysql and sqlalchemy as you have learnt in the lesson of importing/exporting data \n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "PF9YpOvj2UQe",
+ "outputId": "a32ac729-2167-4405-b869-526d56509b9a"
+ },
+ "source": [
+ "!pip install pymysql"
+ ],
+ "execution_count": 4,
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "Collecting pymysql\n",
+ " Downloading PyMySQL-1.0.2-py3-none-any.whl (43 kB)\n",
+ "\u001b[?25l\r\u001b[K |███████▌ | 10 kB 28.5 MB/s eta 0:00:01\r\u001b[K |███████████████ | 20 kB 36.9 MB/s eta 0:00:01\r\u001b[K |██████████████████████▍ | 30 kB 27.3 MB/s eta 0:00:01\r\u001b[K |██████████████████████████████ | 40 kB 20.6 MB/s eta 0:00:01\r\u001b[K |████████████████████████████████| 43 kB 1.0 MB/s \n",
+ "\u001b[?25hInstalling collected packages: pymysql\n",
+ "Successfully installed pymysql-1.0.2\n"
+ ]
+ }
+ ]
+ },
+ {
+ "cell_type": "code",
+ "metadata": {
+ "id": "SXoRote33wxd"
+ },
+ "source": [
+ "import pymysql\n",
+ "from sqlalchemy import create_engine"
+ ],
+ "execution_count": 5,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "wUYXOW7P2UQf"
+ },
+ "source": [
+ "#### 3. Create a mysql engine to set the connection to the server. Check the connection details in [this link](https://relational.fit.cvut.cz/search?tableCount%5B%5D=0-10&tableCount%5B%5D=10-30&dataType%5B%5D=Numeric&databaseSize%5B%5D=KB&databaseSize%5B%5D=MB)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "metadata": {
+ "id": "xgwsLgz62UQf"
+ },
+ "source": [
+ "engine = create_engine('mysql+pymysql://guest:relational@relational.fit.cvut.cz/stats')"
+ ],
+ "execution_count": 6,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "UeJ5RVC12UQh"
+ },
+ "source": [
+ "#### 4. Import the users table "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 348
+ },
+ "id": "LQID8hnV2UQi",
+ "outputId": "6aa3b4bc-e3f3-499e-ec14-0f0751c2ca56"
+ },
+ "source": [
+ "data = pd.read_sql_query('SELECT * FROM stats.users', engine)\n",
+ "data.head()"
+ ],
+ "execution_count": 7,
+ "outputs": [
+ {
+ "output_type": "execute_result",
+ "data": {
+ "text/html": [
+ "
\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " | \n",
+ " Id | \n",
+ " Reputation | \n",
+ " CreationDate | \n",
+ " DisplayName | \n",
+ " LastAccessDate | \n",
+ " WebsiteUrl | \n",
+ " Location | \n",
+ " AboutMe | \n",
+ " Views | \n",
+ " UpVotes | \n",
+ " DownVotes | \n",
+ " AccountId | \n",
+ " Age | \n",
+ " ProfileImageUrl | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " | 0 | \n",
+ " -1 | \n",
+ " 1 | \n",
+ " 2010-07-19 06:55:26 | \n",
+ " Community | \n",
+ " 2010-07-19 06:55:26 | \n",
+ " http://meta.stackexchange.com/ | \n",
+ " on the server farm | \n",
+ " <p>Hi, I'm not really a person.</p>\\n\\n<p>I'm ... | \n",
+ " 0 | \n",
+ " 5007 | \n",
+ " 1920 | \n",
+ " -1 | \n",
+ " NaN | \n",
+ " None | \n",
+ "
\n",
+ " \n",
+ " | 1 | \n",
+ " 2 | \n",
+ " 101 | \n",
+ " 2010-07-19 14:01:36 | \n",
+ " Geoff Dalgas | \n",
+ " 2013-11-12 22:07:23 | \n",
+ " http://stackoverflow.com | \n",
+ " Corvallis, OR | \n",
+ " <p>Developer on the StackOverflow team. Find ... | \n",
+ " 25 | \n",
+ " 3 | \n",
+ " 0 | \n",
+ " 2 | \n",
+ " 37.0 | \n",
+ " None | \n",
+ "
\n",
+ " \n",
+ " | 2 | \n",
+ " 3 | \n",
+ " 101 | \n",
+ " 2010-07-19 15:34:50 | \n",
+ " Jarrod Dixon | \n",
+ " 2014-08-08 06:42:58 | \n",
+ " http://stackoverflow.com | \n",
+ " New York, NY | \n",
+ " <p><a href=\"http://blog.stackoverflow.com/2009... | \n",
+ " 22 | \n",
+ " 19 | \n",
+ " 0 | \n",
+ " 3 | \n",
+ " 35.0 | \n",
+ " None | \n",
+ "
\n",
+ " \n",
+ " | 3 | \n",
+ " 4 | \n",
+ " 101 | \n",
+ " 2010-07-19 19:03:27 | \n",
+ " Emmett | \n",
+ " 2014-01-02 09:31:02 | \n",
+ " http://minesweeperonline.com | \n",
+ " San Francisco, CA | \n",
+ " <p>currently at a startup in SF</p>\\n\\n<p>form... | \n",
+ " 11 | \n",
+ " 0 | \n",
+ " 0 | \n",
+ " 1998 | \n",
+ " 28.0 | \n",
+ " http://i.stack.imgur.com/d1oHX.jpg | \n",
+ "
\n",
+ " \n",
+ " | 4 | \n",
+ " 5 | \n",
+ " 6792 | \n",
+ " 2010-07-19 19:03:57 | \n",
+ " Shane | \n",
+ " 2014-08-13 00:23:47 | \n",
+ " http://www.statalgo.com | \n",
+ " New York, NY | \n",
+ " <p>Quantitative researcher focusing on statist... | \n",
+ " 1145 | \n",
+ " 662 | \n",
+ " 5 | \n",
+ " 54503 | \n",
+ " 35.0 | \n",
+ " None | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
"
+ ],
+ "text/plain": [
+ " Id Reputation ... Age ProfileImageUrl\n",
+ "0 -1 1 ... NaN None\n",
+ "1 2 101 ... 37.0 None\n",
+ "2 3 101 ... 35.0 None\n",
+ "3 4 101 ... 28.0 http://i.stack.imgur.com/d1oHX.jpg\n",
+ "4 5 6792 ... 35.0 None\n",
+ "\n",
+ "[5 rows x 14 columns]"
+ ]
+ },
+ "metadata": {},
+ "execution_count": 7
+ }
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "VmXB3dn42UQl"
+ },
+ "source": [
+ "#### 5. Rename Id column to userId"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 348
+ },
+ "id": "uNsH_hvM2UQn",
+ "outputId": "3ab0860a-0faf-4be0-eda0-97fc86ce15ee"
+ },
+ "source": [
+ "users = data.rename(columns={'Id': 'userId'})\n",
+ "users.head()"
+ ],
+ "execution_count": 15,
+ "outputs": [
+ {
+ "output_type": "execute_result",
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " | \n",
+ " userId | \n",
+ " Reputation | \n",
+ " CreationDate | \n",
+ " DisplayName | \n",
+ " LastAccessDate | \n",
+ " WebsiteUrl | \n",
+ " Location | \n",
+ " AboutMe | \n",
+ " Views | \n",
+ " UpVotes | \n",
+ " DownVotes | \n",
+ " AccountId | \n",
+ " Age | \n",
+ " ProfileImageUrl | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " | 0 | \n",
+ " -1 | \n",
+ " 1 | \n",
+ " 2010-07-19 06:55:26 | \n",
+ " Community | \n",
+ " 2010-07-19 06:55:26 | \n",
+ " http://meta.stackexchange.com/ | \n",
+ " on the server farm | \n",
+ " <p>Hi, I'm not really a person.</p>\\n\\n<p>I'm ... | \n",
+ " 0 | \n",
+ " 5007 | \n",
+ " 1920 | \n",
+ " -1 | \n",
+ " NaN | \n",
+ " None | \n",
+ "
\n",
+ " \n",
+ " | 1 | \n",
+ " 2 | \n",
+ " 101 | \n",
+ " 2010-07-19 14:01:36 | \n",
+ " Geoff Dalgas | \n",
+ " 2013-11-12 22:07:23 | \n",
+ " http://stackoverflow.com | \n",
+ " Corvallis, OR | \n",
+ " <p>Developer on the StackOverflow team. Find ... | \n",
+ " 25 | \n",
+ " 3 | \n",
+ " 0 | \n",
+ " 2 | \n",
+ " 37.0 | \n",
+ " None | \n",
+ "
\n",
+ " \n",
+ " | 2 | \n",
+ " 3 | \n",
+ " 101 | \n",
+ " 2010-07-19 15:34:50 | \n",
+ " Jarrod Dixon | \n",
+ " 2014-08-08 06:42:58 | \n",
+ " http://stackoverflow.com | \n",
+ " New York, NY | \n",
+ " <p><a href=\"http://blog.stackoverflow.com/2009... | \n",
+ " 22 | \n",
+ " 19 | \n",
+ " 0 | \n",
+ " 3 | \n",
+ " 35.0 | \n",
+ " None | \n",
+ "
\n",
+ " \n",
+ " | 3 | \n",
+ " 4 | \n",
+ " 101 | \n",
+ " 2010-07-19 19:03:27 | \n",
+ " Emmett | \n",
+ " 2014-01-02 09:31:02 | \n",
+ " http://minesweeperonline.com | \n",
+ " San Francisco, CA | \n",
+ " <p>currently at a startup in SF</p>\\n\\n<p>form... | \n",
+ " 11 | \n",
+ " 0 | \n",
+ " 0 | \n",
+ " 1998 | \n",
+ " 28.0 | \n",
+ " http://i.stack.imgur.com/d1oHX.jpg | \n",
+ "
\n",
+ " \n",
+ " | 4 | \n",
+ " 5 | \n",
+ " 6792 | \n",
+ " 2010-07-19 19:03:57 | \n",
+ " Shane | \n",
+ " 2014-08-13 00:23:47 | \n",
+ " http://www.statalgo.com | \n",
+ " New York, NY | \n",
+ " <p>Quantitative researcher focusing on statist... | \n",
+ " 1145 | \n",
+ " 662 | \n",
+ " 5 | \n",
+ " 54503 | \n",
+ " 35.0 | \n",
+ " None | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
"
+ ],
+ "text/plain": [
+ " userId Reputation ... Age ProfileImageUrl\n",
+ "0 -1 1 ... NaN None\n",
+ "1 2 101 ... 37.0 None\n",
+ "2 3 101 ... 35.0 None\n",
+ "3 4 101 ... 28.0 http://i.stack.imgur.com/d1oHX.jpg\n",
+ "4 5 6792 ... 35.0 None\n",
+ "\n",
+ "[5 rows x 14 columns]"
+ ]
+ },
+ "metadata": {},
+ "execution_count": 15
+ }
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "W0o0mBZw2UQo"
+ },
+ "source": [
+ "#### 6. Import the posts table. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 521
+ },
+ "id": "JkykFXU92UQo",
+ "outputId": "dc5d27f5-f769-46e9-9bab-3bcdff324c54"
+ },
+ "source": [
+ "posts = pd.read_sql_query('SELECT * FROM stats.posts', engine)\n",
+ "posts.head()"
+ ],
+ "execution_count": 11,
+ "outputs": [
+ {
+ "output_type": "execute_result",
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " | \n",
+ " Id | \n",
+ " PostTypeId | \n",
+ " AcceptedAnswerId | \n",
+ " CreaionDate | \n",
+ " Score | \n",
+ " ViewCount | \n",
+ " Body | \n",
+ " OwnerUserId | \n",
+ " LasActivityDate | \n",
+ " Title | \n",
+ " Tags | \n",
+ " AnswerCount | \n",
+ " CommentCount | \n",
+ " FavoriteCount | \n",
+ " LastEditorUserId | \n",
+ " LastEditDate | \n",
+ " CommunityOwnedDate | \n",
+ " ParentId | \n",
+ " ClosedDate | \n",
+ " OwnerDisplayName | \n",
+ " LastEditorDisplayName | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " | 0 | \n",
+ " 1 | \n",
+ " 1 | \n",
+ " 15.0 | \n",
+ " 2010-07-19 19:12:12 | \n",
+ " 23 | \n",
+ " 1278.0 | \n",
+ " <p>How should I elicit prior distributions fro... | \n",
+ " 8.0 | \n",
+ " 2010-09-15 21:08:26 | \n",
+ " Eliciting priors from experts | \n",
+ " <bayesian><prior><elicitation> | \n",
+ " 5.0 | \n",
+ " 1 | \n",
+ " 14.0 | \n",
+ " NaN | \n",
+ " NaT | \n",
+ " NaT | \n",
+ " NaN | \n",
+ " NaT | \n",
+ " None | \n",
+ " None | \n",
+ "
\n",
+ " \n",
+ " | 1 | \n",
+ " 2 | \n",
+ " 1 | \n",
+ " 59.0 | \n",
+ " 2010-07-19 19:12:57 | \n",
+ " 22 | \n",
+ " 8198.0 | \n",
+ " <p>In many different statistical methods there... | \n",
+ " 24.0 | \n",
+ " 2012-11-12 09:21:54 | \n",
+ " What is normality? | \n",
+ " <distributions><normality> | \n",
+ " 7.0 | \n",
+ " 1 | \n",
+ " 8.0 | \n",
+ " 88.0 | \n",
+ " 2010-08-07 17:56:44 | \n",
+ " NaT | \n",
+ " NaN | \n",
+ " NaT | \n",
+ " None | \n",
+ " None | \n",
+ "
\n",
+ " \n",
+ " | 2 | \n",
+ " 3 | \n",
+ " 1 | \n",
+ " 5.0 | \n",
+ " 2010-07-19 19:13:28 | \n",
+ " 54 | \n",
+ " 3613.0 | \n",
+ " <p>What are some valuable Statistical Analysis... | \n",
+ " 18.0 | \n",
+ " 2013-05-27 14:48:36 | \n",
+ " What are some valuable Statistical Analysis op... | \n",
+ " <software><open-source> | \n",
+ " 19.0 | \n",
+ " 4 | \n",
+ " 36.0 | \n",
+ " 183.0 | \n",
+ " 2011-02-12 05:50:03 | \n",
+ " 2010-07-19 19:13:28 | \n",
+ " NaN | \n",
+ " NaT | \n",
+ " None | \n",
+ " None | \n",
+ "
\n",
+ " \n",
+ " | 3 | \n",
+ " 4 | \n",
+ " 1 | \n",
+ " 135.0 | \n",
+ " 2010-07-19 19:13:31 | \n",
+ " 13 | \n",
+ " 5224.0 | \n",
+ " <p>I have two groups of data. Each with a dif... | \n",
+ " 23.0 | \n",
+ " 2010-09-08 03:00:19 | \n",
+ " Assessing the significance of differences in d... | \n",
+ " <distributions><statistical-significance> | \n",
+ " 5.0 | \n",
+ " 2 | \n",
+ " 2.0 | \n",
+ " NaN | \n",
+ " NaT | \n",
+ " NaT | \n",
+ " NaN | \n",
+ " NaT | \n",
+ " None | \n",
+ " None | \n",
+ "
\n",
+ " \n",
+ " | 4 | \n",
+ " 5 | \n",
+ " 2 | \n",
+ " NaN | \n",
+ " 2010-07-19 19:14:43 | \n",
+ " 81 | \n",
+ " NaN | \n",
+ " <p>The R-project</p>\\n\\n<p><a href=\"http://www... | \n",
+ " 23.0 | \n",
+ " 2010-07-19 19:21:15 | \n",
+ " None | \n",
+ " None | \n",
+ " NaN | \n",
+ " 3 | \n",
+ " NaN | \n",
+ " 23.0 | \n",
+ " 2010-07-19 19:21:15 | \n",
+ " 2010-07-19 19:14:43 | \n",
+ " 3.0 | \n",
+ " NaT | \n",
+ " None | \n",
+ " None | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
"
+ ],
+ "text/plain": [
+ " Id PostTypeId ... OwnerDisplayName LastEditorDisplayName\n",
+ "0 1 1 ... None None\n",
+ "1 2 1 ... None None\n",
+ "2 3 1 ... None None\n",
+ "3 4 1 ... None None\n",
+ "4 5 2 ... None None\n",
+ "\n",
+ "[5 rows x 21 columns]"
+ ]
+ },
+ "metadata": {},
+ "execution_count": 11
+ }
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "og_YRbks2UQp"
+ },
+ "source": [
+ "#### 7. Rename Id column to postId and OwnerUserId to userId"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 521
+ },
+ "id": "ZxnAMcPp2UQq",
+ "outputId": "bb17f0dc-76f1-465a-cf73-ec975fc2c25d"
+ },
+ "source": [
+ "posts2 = posts.rename(columns={'Id': 'postId', 'OwnerUserId': 'userId'}).head()\n",
+ "posts2.head()"
+ ],
+ "execution_count": 16,
+ "outputs": [
+ {
+ "output_type": "execute_result",
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " | \n",
+ " postId | \n",
+ " PostTypeId | \n",
+ " AcceptedAnswerId | \n",
+ " CreaionDate | \n",
+ " Score | \n",
+ " ViewCount | \n",
+ " Body | \n",
+ " userId | \n",
+ " LasActivityDate | \n",
+ " Title | \n",
+ " Tags | \n",
+ " AnswerCount | \n",
+ " CommentCount | \n",
+ " FavoriteCount | \n",
+ " LastEditorUserId | \n",
+ " LastEditDate | \n",
+ " CommunityOwnedDate | \n",
+ " ParentId | \n",
+ " ClosedDate | \n",
+ " OwnerDisplayName | \n",
+ " LastEditorDisplayName | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " | 0 | \n",
+ " 1 | \n",
+ " 1 | \n",
+ " 15.0 | \n",
+ " 2010-07-19 19:12:12 | \n",
+ " 23 | \n",
+ " 1278.0 | \n",
+ " <p>How should I elicit prior distributions fro... | \n",
+ " 8.0 | \n",
+ " 2010-09-15 21:08:26 | \n",
+ " Eliciting priors from experts | \n",
+ " <bayesian><prior><elicitation> | \n",
+ " 5.0 | \n",
+ " 1 | \n",
+ " 14.0 | \n",
+ " NaN | \n",
+ " NaT | \n",
+ " NaT | \n",
+ " NaN | \n",
+ " NaT | \n",
+ " None | \n",
+ " None | \n",
+ "
\n",
+ " \n",
+ " | 1 | \n",
+ " 2 | \n",
+ " 1 | \n",
+ " 59.0 | \n",
+ " 2010-07-19 19:12:57 | \n",
+ " 22 | \n",
+ " 8198.0 | \n",
+ " <p>In many different statistical methods there... | \n",
+ " 24.0 | \n",
+ " 2012-11-12 09:21:54 | \n",
+ " What is normality? | \n",
+ " <distributions><normality> | \n",
+ " 7.0 | \n",
+ " 1 | \n",
+ " 8.0 | \n",
+ " 88.0 | \n",
+ " 2010-08-07 17:56:44 | \n",
+ " NaT | \n",
+ " NaN | \n",
+ " NaT | \n",
+ " None | \n",
+ " None | \n",
+ "
\n",
+ " \n",
+ " | 2 | \n",
+ " 3 | \n",
+ " 1 | \n",
+ " 5.0 | \n",
+ " 2010-07-19 19:13:28 | \n",
+ " 54 | \n",
+ " 3613.0 | \n",
+ " <p>What are some valuable Statistical Analysis... | \n",
+ " 18.0 | \n",
+ " 2013-05-27 14:48:36 | \n",
+ " What are some valuable Statistical Analysis op... | \n",
+ " <software><open-source> | \n",
+ " 19.0 | \n",
+ " 4 | \n",
+ " 36.0 | \n",
+ " 183.0 | \n",
+ " 2011-02-12 05:50:03 | \n",
+ " 2010-07-19 19:13:28 | \n",
+ " NaN | \n",
+ " NaT | \n",
+ " None | \n",
+ " None | \n",
+ "
\n",
+ " \n",
+ " | 3 | \n",
+ " 4 | \n",
+ " 1 | \n",
+ " 135.0 | \n",
+ " 2010-07-19 19:13:31 | \n",
+ " 13 | \n",
+ " 5224.0 | \n",
+ " <p>I have two groups of data. Each with a dif... | \n",
+ " 23.0 | \n",
+ " 2010-09-08 03:00:19 | \n",
+ " Assessing the significance of differences in d... | \n",
+ " <distributions><statistical-significance> | \n",
+ " 5.0 | \n",
+ " 2 | \n",
+ " 2.0 | \n",
+ " NaN | \n",
+ " NaT | \n",
+ " NaT | \n",
+ " NaN | \n",
+ " NaT | \n",
+ " None | \n",
+ " None | \n",
+ "
\n",
+ " \n",
+ " | 4 | \n",
+ " 5 | \n",
+ " 2 | \n",
+ " NaN | \n",
+ " 2010-07-19 19:14:43 | \n",
+ " 81 | \n",
+ " NaN | \n",
+ " <p>The R-project</p>\\n\\n<p><a href=\"http://www... | \n",
+ " 23.0 | \n",
+ " 2010-07-19 19:21:15 | \n",
+ " None | \n",
+ " None | \n",
+ " NaN | \n",
+ " 3 | \n",
+ " NaN | \n",
+ " 23.0 | \n",
+ " 2010-07-19 19:21:15 | \n",
+ " 2010-07-19 19:14:43 | \n",
+ " 3.0 | \n",
+ " NaT | \n",
+ " None | \n",
+ " None | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
"
+ ],
+ "text/plain": [
+ " postId PostTypeId ... OwnerDisplayName LastEditorDisplayName\n",
+ "0 1 1 ... None None\n",
+ "1 2 1 ... None None\n",
+ "2 3 1 ... None None\n",
+ "3 4 1 ... None None\n",
+ "4 5 2 ... None None\n",
+ "\n",
+ "[5 rows x 21 columns]"
+ ]
+ },
+ "metadata": {},
+ "execution_count": 16
+ }
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "f7KS0pwd2UQq"
+ },
+ "source": [
+ "#### 8. Define new dataframes for users and posts with the following selected columns:\n",
+ " **users columns**: userId, Reputation,Views,UpVotes,DownVotes\n",
+ " **posts columns**: postId, Score,userID,ViewCount,CommentCount"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 206
+ },
+ "id": "z6wGEl4h2UQr",
+ "outputId": "ac0c93c8-a232-4eb1-9451-4fa985b2f56d"
+ },
+ "source": [
+ "users_filt = users[['userId', 'Reputation', 'Views', 'UpVotes', 'DownVotes']]\n",
+ "posts_filt = posts2[['postId', 'Score', 'userId', 'ViewCount', 'CommentCount']]\n",
+ "posts_filt.head()"
+ ],
+ "execution_count": 20,
+ "outputs": [
+ {
+ "output_type": "execute_result",
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " | \n",
+ " postId | \n",
+ " Score | \n",
+ " userId | \n",
+ " ViewCount | \n",
+ " CommentCount | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " | 0 | \n",
+ " 1 | \n",
+ " 23 | \n",
+ " 8.0 | \n",
+ " 1278.0 | \n",
+ " 1 | \n",
+ "
\n",
+ " \n",
+ " | 1 | \n",
+ " 2 | \n",
+ " 22 | \n",
+ " 24.0 | \n",
+ " 8198.0 | \n",
+ " 1 | \n",
+ "
\n",
+ " \n",
+ " | 2 | \n",
+ " 3 | \n",
+ " 54 | \n",
+ " 18.0 | \n",
+ " 3613.0 | \n",
+ " 4 | \n",
+ "
\n",
+ " \n",
+ " | 3 | \n",
+ " 4 | \n",
+ " 13 | \n",
+ " 23.0 | \n",
+ " 5224.0 | \n",
+ " 2 | \n",
+ "
\n",
+ " \n",
+ " | 4 | \n",
+ " 5 | \n",
+ " 81 | \n",
+ " 23.0 | \n",
+ " NaN | \n",
+ " 3 | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
"
+ ],
+ "text/plain": [
+ " postId Score userId ViewCount CommentCount\n",
+ "0 1 23 8.0 1278.0 1\n",
+ "1 2 22 24.0 8198.0 1\n",
+ "2 3 54 18.0 3613.0 4\n",
+ "3 4 13 23.0 5224.0 2\n",
+ "4 5 81 23.0 NaN 3"
+ ]
+ },
+ "metadata": {},
+ "execution_count": 20
+ }
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "2R_15Bkt2UQr"
+ },
+ "source": [
+ "#### 8. Merge both dataframes, users and posts. \n",
+ "You will need to make a [merge](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.merge.html) of posts and users dataframes."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 206
+ },
+ "id": "szphPZCK2UQs",
+ "outputId": "4aa2304e-8dd4-4dd4-9062-08cbaebdbecc"
+ },
+ "source": [
+ "merged_df = pd.merge(left=users_filt, right=posts_filt, left_on='userId', right_on='userId')\n",
+ "merged_df.head()"
+ ],
+ "execution_count": 31,
+ "outputs": [
+ {
+ "output_type": "execute_result",
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " | \n",
+ " userId | \n",
+ " Reputation | \n",
+ " Views | \n",
+ " UpVotes | \n",
+ " DownVotes | \n",
+ " postId | \n",
+ " Score | \n",
+ " ViewCount | \n",
+ " CommentCount | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " | 0 | \n",
+ " 8 | \n",
+ " 6764 | \n",
+ " 1089 | \n",
+ " 604 | \n",
+ " 25 | \n",
+ " 1 | \n",
+ " 23 | \n",
+ " 1278.0 | \n",
+ " 1 | \n",
+ "
\n",
+ " \n",
+ " | 1 | \n",
+ " 18 | \n",
+ " 128 | \n",
+ " 8 | \n",
+ " 16 | \n",
+ " 0 | \n",
+ " 3 | \n",
+ " 54 | \n",
+ " 3613.0 | \n",
+ " 4 | \n",
+ "
\n",
+ " \n",
+ " | 2 | \n",
+ " 23 | \n",
+ " 308 | \n",
+ " 52 | \n",
+ " 34 | \n",
+ " 1 | \n",
+ " 4 | \n",
+ " 13 | \n",
+ " 5224.0 | \n",
+ " 2 | \n",
+ "
\n",
+ " \n",
+ " | 3 | \n",
+ " 23 | \n",
+ " 308 | \n",
+ " 52 | \n",
+ " 34 | \n",
+ " 1 | \n",
+ " 5 | \n",
+ " 81 | \n",
+ " NaN | \n",
+ " 3 | \n",
+ "
\n",
+ " \n",
+ " | 4 | \n",
+ " 24 | \n",
+ " 344 | \n",
+ " 48 | \n",
+ " 36 | \n",
+ " 1 | \n",
+ " 2 | \n",
+ " 22 | \n",
+ " 8198.0 | \n",
+ " 1 | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
"
+ ],
+ "text/plain": [
+ " userId Reputation Views UpVotes ... postId Score ViewCount CommentCount\n",
+ "0 8 6764 1089 604 ... 1 23 1278.0 1\n",
+ "1 18 128 8 16 ... 3 54 3613.0 4\n",
+ "2 23 308 52 34 ... 4 13 5224.0 2\n",
+ "3 23 308 52 34 ... 5 81 NaN 3\n",
+ "4 24 344 48 36 ... 2 22 8198.0 1\n",
+ "\n",
+ "[5 rows x 9 columns]"
+ ]
+ },
+ "metadata": {},
+ "execution_count": 31
+ }
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "jz7i0OjN2UQs"
+ },
+ "source": [
+ "#### 9. How many missing values do you have in your merged dataframe? On which columns?"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "metadata": {
+ "id": "YsRGHRgqHsWs"
+ },
+ "source": [
+ "users.info()"
+ ],
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "4Xs_xbvd2UQs",
+ "outputId": "2a3cb280-b23b-4995-81f0-f8f1a40c335b"
+ },
+ "source": [
+ "merged_df.info()"
+ ],
+ "execution_count": 32,
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "\n",
+ "Int64Index: 5 entries, 0 to 4\n",
+ "Data columns (total 9 columns):\n",
+ " # Column Non-Null Count Dtype \n",
+ "--- ------ -------------- ----- \n",
+ " 0 userId 5 non-null int64 \n",
+ " 1 Reputation 5 non-null int64 \n",
+ " 2 Views 5 non-null int64 \n",
+ " 3 UpVotes 5 non-null int64 \n",
+ " 4 DownVotes 5 non-null int64 \n",
+ " 5 postId 5 non-null int64 \n",
+ " 6 Score 5 non-null int64 \n",
+ " 7 ViewCount 4 non-null float64\n",
+ " 8 CommentCount 5 non-null int64 \n",
+ "dtypes: float64(1), int64(8)\n",
+ "memory usage: 400.0 bytes\n"
+ ]
+ }
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "Ys7NnxD92UQt"
+ },
+ "source": [
+ "#### 10. You will need to make something with missing values. Will you clean or filling them? Explain. \n",
+ "**Remember** to check the results of your code before passing to the next step"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "cnv71DTGKNJs"
+ },
+ "source": [
+ "Hay que reemplazar el valor nulo (NaN) por un float para que no haya problema al manipular los valores de la columna"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 206
+ },
+ "id": "umGNR1EF2UQt",
+ "outputId": "f152d2d0-85f4-4b8f-ad78-23c2370432db"
+ },
+ "source": [
+ "merged_df[['ViewCount']] = merged_df[['ViewCount']].fillna(0)\n",
+ "merged_df.head()"
+ ],
+ "execution_count": 39,
+ "outputs": [
+ {
+ "output_type": "execute_result",
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " | \n",
+ " userId | \n",
+ " Reputation | \n",
+ " Views | \n",
+ " UpVotes | \n",
+ " DownVotes | \n",
+ " postId | \n",
+ " Score | \n",
+ " ViewCount | \n",
+ " CommentCount | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " | 0 | \n",
+ " 8 | \n",
+ " 6764 | \n",
+ " 1089 | \n",
+ " 604 | \n",
+ " 25 | \n",
+ " 1 | \n",
+ " 23 | \n",
+ " 1278.0 | \n",
+ " 1 | \n",
+ "
\n",
+ " \n",
+ " | 1 | \n",
+ " 18 | \n",
+ " 128 | \n",
+ " 8 | \n",
+ " 16 | \n",
+ " 0 | \n",
+ " 3 | \n",
+ " 54 | \n",
+ " 3613.0 | \n",
+ " 4 | \n",
+ "
\n",
+ " \n",
+ " | 2 | \n",
+ " 23 | \n",
+ " 308 | \n",
+ " 52 | \n",
+ " 34 | \n",
+ " 1 | \n",
+ " 4 | \n",
+ " 13 | \n",
+ " 5224.0 | \n",
+ " 2 | \n",
+ "
\n",
+ " \n",
+ " | 3 | \n",
+ " 23 | \n",
+ " 308 | \n",
+ " 52 | \n",
+ " 34 | \n",
+ " 1 | \n",
+ " 5 | \n",
+ " 81 | \n",
+ " 0.0 | \n",
+ " 3 | \n",
+ "
\n",
+ " \n",
+ " | 4 | \n",
+ " 24 | \n",
+ " 344 | \n",
+ " 48 | \n",
+ " 36 | \n",
+ " 1 | \n",
+ " 2 | \n",
+ " 22 | \n",
+ " 8198.0 | \n",
+ " 1 | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
"
+ ],
+ "text/plain": [
+ " userId Reputation Views UpVotes ... postId Score ViewCount CommentCount\n",
+ "0 8 6764 1089 604 ... 1 23 1278.0 1\n",
+ "1 18 128 8 16 ... 3 54 3613.0 4\n",
+ "2 23 308 52 34 ... 4 13 5224.0 2\n",
+ "3 23 308 52 34 ... 5 81 0.0 3\n",
+ "4 24 344 48 36 ... 2 22 8198.0 1\n",
+ "\n",
+ "[5 rows x 9 columns]"
+ ]
+ },
+ "metadata": {},
+ "execution_count": 39
+ }
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "epUVhwyo2UQt"
+ },
+ "source": [
+ "#### 11. Adjust the data types in order to avoid future issues. Which ones should be changed? "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "metadata": {
+ "id": "XxTKlS_82UQt"
+ },
+ "source": [
+ ""
+ ],
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "kJZvR7cc2UQu"
+ },
+ "source": [
+ "#### Bonus: Identify extreme values in your merged dataframe as you have learned in class, create a dataframe called outliers with the same columns as our data set and calculate the bounds. The values of the outliers dataframe will be the values of the merged_df that fall outside that bounds. You will need to save your outliers dataframe to a csv file on your-code folder."
+ ]
+ }
+ ]
+}
\ No newline at end of file
diff --git a/your-code/main.ipynb b/your-code/main.ipynb
deleted file mode 100644
index 31724c5..0000000
--- a/your-code/main.ipynb
+++ /dev/null
@@ -1,204 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "#### 1. Import pandas library"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": []
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "#### 2. Import pymysql and sqlalchemy as you have learnt in the lesson of importing/exporting data \n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": []
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "#### 3. Create a mysql engine to set the connection to the server. Check the connection details in [this link](https://relational.fit.cvut.cz/search?tableCount%5B%5D=0-10&tableCount%5B%5D=10-30&dataType%5B%5D=Numeric&databaseSize%5B%5D=KB&databaseSize%5B%5D=MB)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": []
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "#### 4. Import the users table "
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": []
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "#### 5. Rename Id column to userId"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": []
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "#### 6. Import the posts table. "
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 7,
- "metadata": {},
- "outputs": [],
- "source": []
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "#### 7. Rename Id column to postId and OwnerUserId to userId"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 8,
- "metadata": {},
- "outputs": [],
- "source": []
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "#### 8. Define new dataframes for users and posts with the following selected columns:\n",
- " **users columns**: userId, Reputation,Views,UpVotes,DownVotes\n",
- " **posts columns**: postId, Score,userID,ViewCount,CommentCount"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": []
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "#### 8. Merge both dataframes, users and posts. \n",
- "You will need to make a [merge](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.merge.html) of posts and users dataframes."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 11,
- "metadata": {},
- "outputs": [],
- "source": []
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "#### 9. How many missing values do you have in your merged dataframe? On which columns?"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": []
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "#### 10. You will need to make something with missing values. Will you clean or filling them? Explain. \n",
- "**Remember** to check the results of your code before passing to the next step"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": []
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "#### 11. Adjust the data types in order to avoid future issues. Which ones should be changed? "
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": []
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "#### Bonus: Identify extreme values in your merged dataframe as you have learned in class, create a dataframe called outliers with the same columns as our data set and calculate the bounds. The values of the outliers dataframe will be the values of the merged_df that fall outside that bounds. You will need to save your outliers dataframe to a csv file on your-code folder."
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": "Python 3",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.6.5"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}