diff --git a/.DS_Store b/.DS_Store new file mode 100644 index 0000000..3d23cb3 Binary files /dev/null and b/.DS_Store differ diff --git a/.ipynb_checkpoints/Solutions-checkpoint.ipynb b/.ipynb_checkpoints/Solutions-checkpoint.ipynb new file mode 100644 index 0000000..7c6e66e --- /dev/null +++ b/.ipynb_checkpoints/Solutions-checkpoint.ipynb @@ -0,0 +1,589 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "ee314908", + "metadata": {}, + "source": [ + "# Lab | Inferential statistics - T-test & P-value" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "id": "cafe0ae7-2c1f-4bfb-b8ce-91ec5f88a309", + "metadata": {}, + "outputs": [], + "source": [ + "import pandas as pd\n", + "import numpy as np\n", + "import math\n", + "import statistics\n", + "from scipy.stats import ttest_ind, norm" + ] + }, + { + "cell_type": "markdown", + "id": "515aa25e", + "metadata": {}, + "source": [ + "## 1. We will have another simple example on two sample t test (pooled- when the variances are equal). But this time this is a one sided t-test\n", + "\n", + "In a packing plant, a machine packs cartons with jars. It is supposed that a new machine will pack faster on the average than the machine currently used. To test that hypothesis, the times it takes each machine to pack ten cartons are recorded. The results, in seconds, are shown in the tables in the file `files_for_lab/machine.txt`.\n", + "Assume that there is sufficient evidence to conduct the t test, does the data provide sufficient evidence to show if one machine is better than the other\n", + "\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "id": "024e25df-ad3f-41b9-9db0-d90a8cc6b12d", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " new_machine old_machine\n", + "0 42.1 42.7\n", + "1 41.0 43.6\n", + "2 41.3 43.8\n", + "3 41.8 43.3\n", + "4 42.4 42.5\n", + "5 42.8 43.5\n", + "6 43.2 43.1\n", + "7 42.3 41.7\n", + "8 41.8 44.0\n", + "9 42.7 44.1\n" + ] + } + ], + "source": [ + "df = pd.read_excel('/Users/spicasumampouw/IronSpica/lab/Unit_7/lab-t-tests-p-values/files_for_lab/machine.xlsx')\n", + "print(df)" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "id": "c3dfe8ce-f6c7-4573-a71d-d2a88bd93068", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
countmeanstdmin25%50%75%max
new_machine10.042.140.68345541.041.842.242.62543.2
old_machine10.043.230.74988941.742.843.443.75044.1
\n", + "
" + ], + "text/plain": [ + " count mean std min 25% 50% 75% max\n", + "new_machine 10.0 42.14 0.683455 41.0 41.8 42.2 42.625 43.2\n", + "old_machine 10.0 43.23 0.749889 41.7 42.8 43.4 43.750 44.1" + ] + }, + "execution_count": 17, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df.describe().T" + ] + }, + { + "cell_type": "code", + "execution_count": 61, + "id": "f6a79e0b-0993-414a-aa32-1a1f4ea38b98", + "metadata": {}, + "outputs": [], + "source": [ + "sample_mean1 = df[\"new_machine\"].mean()\n", + "sample_mean2 = df[\"old_machine\"].mean()\n", + "sample_std1 = df[\"new_machine\"].std()\n", + "sample_std2 = df[\"old_machine\"].std()\n", + "n1 = len(df[\"new_machine\"])\n", + "n2 = len(df[\"old_machine\"])\n", + "\n", + "pop_mean = df[\"new_machine\"].mean()+df[\"old_machine\"].mean()\n", + "pop_std = df[\"new_machine\"].std()+df[\"old_machine\"].std()\n", + "n = len(df[\"new_machine\"])+len(df[\"old_machine\"])\n", + "\n", + "statistic1 = (sample_mean1 - pop_mean)/(pop_std/math.sqrt(n))\n", + "statistic2 = (sample_mean2 - pop_mean)/(pop_std/math.sqrt(n))" + ] + }, + { + "cell_type": "code", + "execution_count": 62, + "id": "2ce0f7f1-7792-4e04-ad59-3360fcf187df", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Statistic1 is: -134.8806821799226\n", + "Statistic2 is: -131.47980446592499\n" + ] + } + ], + "source": [ + "print(\"Statistic1 is: \", statistic1)\n", + "print(\"Statistic2 is: \", statistic2)" + ] + }, + { + "cell_type": "code", + "execution_count": 63, + "id": "16b21ac4-d88c-4537-88b1-286d099fe4c3", + "metadata": {}, + "outputs": [], + "source": [ + "result_nm = norm.rvs(loc=sample_mean1, scale=sample_std1, size=n1)\n", + "result_om = norm.rvs(loc=sample_mean2, scale=sample_std2, size=n2)" + ] + }, + { + "cell_type": "code", + "execution_count": 64, + "id": "4f263201-4c2d-4c8f-b6c6-00738823dbf2", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[41.16912636 42.7804341 41.83049697 41.84900104 41.42524585 43.18153511\n", + " 41.865567 42.40788018 42.23398789 42.2897537 ] \n", + "\n", + "[41.1795803 42.57820968 43.8178388 42.7216277 42.59886934 44.33333382\n", + " 43.12462427 44.19597322 42.80296527 42.53386869]\n" + ] + } + ], + "source": [ + "print(result_nm, \"\\n\")\n", + "print(result_om)" + ] + }, + { + "cell_type": "code", + "execution_count": 66, + "id": "b209d0ad-0fc1-4871-8622-e829c12d7229", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "We accept the null hypothesis because: Ttest_indResult(statistic=-2.5130521559851178, pvalue=0.021710467320837187).\n", + "And we can say with high level of confidence that we have enough evidence to consider the null hypothesis.\n" + ] + } + ], + "source": [ + "print(f\"We accept the null hypothesis because: {ttest_ind(result_nm, result_om)}.\")\n", + "print(\"And we can say with high level of confidence that we have enough evidence to consider the null hypothesis.\")" + ] + }, + { + "cell_type": "markdown", + "id": "501614ae", + "metadata": {}, + "source": [ + "## 2. An additional problem (not mandatory): In this case we can't assume that the population variances are equal. Hence in this case we cannot pool the variances.\n", + " Independent random samples of 17 sophomores and 13 juniors attending a large university yield the following data on grade point averages. Data is provided in the file `files_for_lab/student_gpa.txt`.\n", + " At the 5% significance level, do the data provide sufficient evidence to conclude that the mean GPAs of sophomores and juniors at the university differ?\n", + "\n", + " Test statistics can be calculated as: [link to the image - Test statistics calculation for Unpooled Variance Case](https://education-team-2020.s3-eu-west-1.amazonaws.com/data-analytics/7.04/7.04-unpooled_variances.png)\n", + "\n", + " Degrees of freedom is `(n1-1)+(n2-1)`." + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "id": "3925f57e", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
sophomoresjuniorsUnnamed: 2
03.042.56NaN
11.712.77NaN
23.302.70NaN
32.883.00NaN
42.112.98NaN
52.603.47NaN
62.923.26NaN
73.603.20NaN
82.283.19NaN
92.822.65NaN
103.033.00NaN
113.133.39NaN
122.862.58NaN
133.49NaNNaN
143.11NaNNaN
152.13NaNNaN
163.27NaNNaN
\n", + "
" + ], + "text/plain": [ + " sophomores juniors Unnamed: 2\n", + "0 3.04 2.56 NaN\n", + "1 1.71 2.77 NaN\n", + "2 3.30 2.70 NaN\n", + "3 2.88 3.00 NaN\n", + "4 2.11 2.98 NaN\n", + "5 2.60 3.47 NaN\n", + "6 2.92 3.26 NaN\n", + "7 3.60 3.20 NaN\n", + "8 2.28 3.19 NaN\n", + "9 2.82 2.65 NaN\n", + "10 3.03 3.00 NaN\n", + "11 3.13 3.39 NaN\n", + "12 2.86 2.58 NaN\n", + "13 3.49 NaN NaN\n", + "14 3.11 NaN NaN\n", + "15 2.13 NaN NaN\n", + "16 3.27 NaN NaN" + ] + }, + "execution_count": 23, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df1 =pd.read_excel(\"/Users/spicasumampouw/IronSpica/lab/Unit_7/lab-t-tests-p-values/files_for_lab/student_gpa.xlsx\")\n", + "df1" + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "id": "139519e0-ba6f-4487-ae93-7301f3a9991a", + "metadata": {}, + "outputs": [], + "source": [ + "df1 = df1.drop(columns=\"Unnamed: 2\")" + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "id": "1e3ecc71-67c2-4511-978d-c8d03f96d178", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
countmeanstdmin25%50%75%max
sophomores17.02.8400000.5198321.712.62.923.133.60
juniors13.02.9807690.3092592.562.73.003.203.47
\n", + "
" + ], + "text/plain": [ + " count mean std min 25% 50% 75% max\n", + "sophomores 17.0 2.840000 0.519832 1.71 2.6 2.92 3.13 3.60\n", + "juniors 13.0 2.980769 0.309259 2.56 2.7 3.00 3.20 3.47" + ] + }, + "execution_count": 25, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df1.describe().T" + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "id": "338b9c92-043a-4997-a1ba-b98742bf03da", + "metadata": {}, + "outputs": [], + "source": [ + "sample_mean_ex1 = df1[\"sophomores\"].mean()\n", + "sample_mean_ex2 = df1[\"juniors\"].mean()\n", + "sample_std_ex1 = df1[\"sophomores\"].std()\n", + "sample_std_ex2 = df1[\"juniors\"].std()\n", + "n_ex1 = len(df1[\"sophomores\"])\n", + "n_ex2 = len(df1[\"juniors\"])" + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "id": "e5a0c297-a0b3-4881-b81c-373ae016523f", + "metadata": {}, + "outputs": [], + "source": [ + "result_s = norm.rvs(loc=sample_mean_ex1, scale=sample_std_ex1, size=n_ex1)\n", + "result_j = norm.rvs(loc=sample_mean_ex2, scale=sample_std_ex2, size=n_ex2)" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "id": "b1e467bd-efdd-4068-b741-370875538cbe", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "We accept the null hypothesis because: Ttest_indResult(statistic=-2.0150645185024647, pvalue=0.052363606866640794).\n" + ] + } + ], + "source": [ + "print(f\"We accept the null hypothesis because: {ttest_ind(result_s, result_j)}.\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "65fde408-f2c3-4a46-a59d-7156d7e3868e", + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.8" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/Solutions.ipynb b/Solutions.ipynb new file mode 100644 index 0000000..7c6e66e --- /dev/null +++ b/Solutions.ipynb @@ -0,0 +1,589 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "ee314908", + "metadata": {}, + "source": [ + "# Lab | Inferential statistics - T-test & P-value" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "id": "cafe0ae7-2c1f-4bfb-b8ce-91ec5f88a309", + "metadata": {}, + "outputs": [], + "source": [ + "import pandas as pd\n", + "import numpy as np\n", + "import math\n", + "import statistics\n", + "from scipy.stats import ttest_ind, norm" + ] + }, + { + "cell_type": "markdown", + "id": "515aa25e", + "metadata": {}, + "source": [ + "## 1. We will have another simple example on two sample t test (pooled- when the variances are equal). But this time this is a one sided t-test\n", + "\n", + "In a packing plant, a machine packs cartons with jars. It is supposed that a new machine will pack faster on the average than the machine currently used. To test that hypothesis, the times it takes each machine to pack ten cartons are recorded. The results, in seconds, are shown in the tables in the file `files_for_lab/machine.txt`.\n", + "Assume that there is sufficient evidence to conduct the t test, does the data provide sufficient evidence to show if one machine is better than the other\n", + "\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "id": "024e25df-ad3f-41b9-9db0-d90a8cc6b12d", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " new_machine old_machine\n", + "0 42.1 42.7\n", + "1 41.0 43.6\n", + "2 41.3 43.8\n", + "3 41.8 43.3\n", + "4 42.4 42.5\n", + "5 42.8 43.5\n", + "6 43.2 43.1\n", + "7 42.3 41.7\n", + "8 41.8 44.0\n", + "9 42.7 44.1\n" + ] + } + ], + "source": [ + "df = pd.read_excel('/Users/spicasumampouw/IronSpica/lab/Unit_7/lab-t-tests-p-values/files_for_lab/machine.xlsx')\n", + "print(df)" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "id": "c3dfe8ce-f6c7-4573-a71d-d2a88bd93068", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
countmeanstdmin25%50%75%max
new_machine10.042.140.68345541.041.842.242.62543.2
old_machine10.043.230.74988941.742.843.443.75044.1
\n", + "
" + ], + "text/plain": [ + " count mean std min 25% 50% 75% max\n", + "new_machine 10.0 42.14 0.683455 41.0 41.8 42.2 42.625 43.2\n", + "old_machine 10.0 43.23 0.749889 41.7 42.8 43.4 43.750 44.1" + ] + }, + "execution_count": 17, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df.describe().T" + ] + }, + { + "cell_type": "code", + "execution_count": 61, + "id": "f6a79e0b-0993-414a-aa32-1a1f4ea38b98", + "metadata": {}, + "outputs": [], + "source": [ + "sample_mean1 = df[\"new_machine\"].mean()\n", + "sample_mean2 = df[\"old_machine\"].mean()\n", + "sample_std1 = df[\"new_machine\"].std()\n", + "sample_std2 = df[\"old_machine\"].std()\n", + "n1 = len(df[\"new_machine\"])\n", + "n2 = len(df[\"old_machine\"])\n", + "\n", + "pop_mean = df[\"new_machine\"].mean()+df[\"old_machine\"].mean()\n", + "pop_std = df[\"new_machine\"].std()+df[\"old_machine\"].std()\n", + "n = len(df[\"new_machine\"])+len(df[\"old_machine\"])\n", + "\n", + "statistic1 = (sample_mean1 - pop_mean)/(pop_std/math.sqrt(n))\n", + "statistic2 = (sample_mean2 - pop_mean)/(pop_std/math.sqrt(n))" + ] + }, + { + "cell_type": "code", + "execution_count": 62, + "id": "2ce0f7f1-7792-4e04-ad59-3360fcf187df", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Statistic1 is: -134.8806821799226\n", + "Statistic2 is: -131.47980446592499\n" + ] + } + ], + "source": [ + "print(\"Statistic1 is: \", statistic1)\n", + "print(\"Statistic2 is: \", statistic2)" + ] + }, + { + "cell_type": "code", + "execution_count": 63, + "id": "16b21ac4-d88c-4537-88b1-286d099fe4c3", + "metadata": {}, + "outputs": [], + "source": [ + "result_nm = norm.rvs(loc=sample_mean1, scale=sample_std1, size=n1)\n", + "result_om = norm.rvs(loc=sample_mean2, scale=sample_std2, size=n2)" + ] + }, + { + "cell_type": "code", + "execution_count": 64, + "id": "4f263201-4c2d-4c8f-b6c6-00738823dbf2", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[41.16912636 42.7804341 41.83049697 41.84900104 41.42524585 43.18153511\n", + " 41.865567 42.40788018 42.23398789 42.2897537 ] \n", + "\n", + "[41.1795803 42.57820968 43.8178388 42.7216277 42.59886934 44.33333382\n", + " 43.12462427 44.19597322 42.80296527 42.53386869]\n" + ] + } + ], + "source": [ + "print(result_nm, \"\\n\")\n", + "print(result_om)" + ] + }, + { + "cell_type": "code", + "execution_count": 66, + "id": "b209d0ad-0fc1-4871-8622-e829c12d7229", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "We accept the null hypothesis because: Ttest_indResult(statistic=-2.5130521559851178, pvalue=0.021710467320837187).\n", + "And we can say with high level of confidence that we have enough evidence to consider the null hypothesis.\n" + ] + } + ], + "source": [ + "print(f\"We accept the null hypothesis because: {ttest_ind(result_nm, result_om)}.\")\n", + "print(\"And we can say with high level of confidence that we have enough evidence to consider the null hypothesis.\")" + ] + }, + { + "cell_type": "markdown", + "id": "501614ae", + "metadata": {}, + "source": [ + "## 2. An additional problem (not mandatory): In this case we can't assume that the population variances are equal. Hence in this case we cannot pool the variances.\n", + " Independent random samples of 17 sophomores and 13 juniors attending a large university yield the following data on grade point averages. Data is provided in the file `files_for_lab/student_gpa.txt`.\n", + " At the 5% significance level, do the data provide sufficient evidence to conclude that the mean GPAs of sophomores and juniors at the university differ?\n", + "\n", + " Test statistics can be calculated as: [link to the image - Test statistics calculation for Unpooled Variance Case](https://education-team-2020.s3-eu-west-1.amazonaws.com/data-analytics/7.04/7.04-unpooled_variances.png)\n", + "\n", + " Degrees of freedom is `(n1-1)+(n2-1)`." + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "id": "3925f57e", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
sophomoresjuniorsUnnamed: 2
03.042.56NaN
11.712.77NaN
23.302.70NaN
32.883.00NaN
42.112.98NaN
52.603.47NaN
62.923.26NaN
73.603.20NaN
82.283.19NaN
92.822.65NaN
103.033.00NaN
113.133.39NaN
122.862.58NaN
133.49NaNNaN
143.11NaNNaN
152.13NaNNaN
163.27NaNNaN
\n", + "
" + ], + "text/plain": [ + " sophomores juniors Unnamed: 2\n", + "0 3.04 2.56 NaN\n", + "1 1.71 2.77 NaN\n", + "2 3.30 2.70 NaN\n", + "3 2.88 3.00 NaN\n", + "4 2.11 2.98 NaN\n", + "5 2.60 3.47 NaN\n", + "6 2.92 3.26 NaN\n", + "7 3.60 3.20 NaN\n", + "8 2.28 3.19 NaN\n", + "9 2.82 2.65 NaN\n", + "10 3.03 3.00 NaN\n", + "11 3.13 3.39 NaN\n", + "12 2.86 2.58 NaN\n", + "13 3.49 NaN NaN\n", + "14 3.11 NaN NaN\n", + "15 2.13 NaN NaN\n", + "16 3.27 NaN NaN" + ] + }, + "execution_count": 23, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df1 =pd.read_excel(\"/Users/spicasumampouw/IronSpica/lab/Unit_7/lab-t-tests-p-values/files_for_lab/student_gpa.xlsx\")\n", + "df1" + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "id": "139519e0-ba6f-4487-ae93-7301f3a9991a", + "metadata": {}, + "outputs": [], + "source": [ + "df1 = df1.drop(columns=\"Unnamed: 2\")" + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "id": "1e3ecc71-67c2-4511-978d-c8d03f96d178", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
countmeanstdmin25%50%75%max
sophomores17.02.8400000.5198321.712.62.923.133.60
juniors13.02.9807690.3092592.562.73.003.203.47
\n", + "
" + ], + "text/plain": [ + " count mean std min 25% 50% 75% max\n", + "sophomores 17.0 2.840000 0.519832 1.71 2.6 2.92 3.13 3.60\n", + "juniors 13.0 2.980769 0.309259 2.56 2.7 3.00 3.20 3.47" + ] + }, + "execution_count": 25, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df1.describe().T" + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "id": "338b9c92-043a-4997-a1ba-b98742bf03da", + "metadata": {}, + "outputs": [], + "source": [ + "sample_mean_ex1 = df1[\"sophomores\"].mean()\n", + "sample_mean_ex2 = df1[\"juniors\"].mean()\n", + "sample_std_ex1 = df1[\"sophomores\"].std()\n", + "sample_std_ex2 = df1[\"juniors\"].std()\n", + "n_ex1 = len(df1[\"sophomores\"])\n", + "n_ex2 = len(df1[\"juniors\"])" + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "id": "e5a0c297-a0b3-4881-b81c-373ae016523f", + "metadata": {}, + "outputs": [], + "source": [ + "result_s = norm.rvs(loc=sample_mean_ex1, scale=sample_std_ex1, size=n_ex1)\n", + "result_j = norm.rvs(loc=sample_mean_ex2, scale=sample_std_ex2, size=n_ex2)" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "id": "b1e467bd-efdd-4068-b741-370875538cbe", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "We accept the null hypothesis because: Ttest_indResult(statistic=-2.0150645185024647, pvalue=0.052363606866640794).\n" + ] + } + ], + "source": [ + "print(f\"We accept the null hypothesis because: {ttest_ind(result_s, result_j)}.\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "65fde408-f2c3-4a46-a59d-7156d7e3868e", + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.8" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/Untitled.ipynb b/Untitled.ipynb deleted file mode 100644 index bb907ae..0000000 --- a/Untitled.ipynb +++ /dev/null @@ -1,76 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "id": "ee314908", - "metadata": {}, - "source": [ - "# Lab | Inferential statistics - T-test & P-value" - ] - }, - { - "cell_type": "markdown", - "id": "515aa25e", - "metadata": {}, - "source": [ - "## 1. We will have another simple example on two sample t test (pooled- when the variances are equal). But this time this is a one sided t-test\n", - "\n", - "In a packing plant, a machine packs cartons with jars. It is supposed that a new machine will pack faster on the average than the machine currently used. To test that hypothesis, the times it takes each machine to pack ten cartons are recorded. The results, in seconds, are shown in the tables in the file `files_for_lab/machine.txt`.\n", - "Assume that there is sufficient evidence to conduct the t test, does the data provide sufficient evidence to show if one machine is better than the other\n", - "\n", - "\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "c4797fc7", - "metadata": {}, - "outputs": [], - "source": [] - }, - { - "cell_type": "markdown", - "id": "501614ae", - "metadata": {}, - "source": [ - "## 2. An additional problem (not mandatory): In this case we can't assume that the population variances are equal. Hence in this case we cannot pool the variances.\n", - " Independent random samples of 17 sophomores and 13 juniors attending a large university yield the following data on grade point averages. Data is provided in the file `files_for_lab/student_gpa.txt`.\n", - " At the 5% significance level, do the data provide sufficient evidence to conclude that the mean GPAs of sophomores and juniors at the university differ?\n", - "\n", - " Test statistics can be calculated as: [link to the image - Test statistics calculation for Unpooled Variance Case](https://education-team-2020.s3-eu-west-1.amazonaws.com/data-analytics/7.04/7.04-unpooled_variances.png)\n", - "\n", - " Degrees of freedom is `(n1-1)+(n2-1)`." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "3925f57e", - "metadata": {}, - "outputs": [], - "source": [] - } - ], - "metadata": { - "kernelspec": { - "display_name": "ironhack", - "language": "python", - "name": "ironhack" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.8.5" - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} diff --git a/files_for_lab/machine.txt b/files_for_lab/machine.txt index b26c8d4..34fc1ec 100644 Binary files a/files_for_lab/machine.txt and b/files_for_lab/machine.txt differ diff --git a/files_for_lab/machine.xlsx b/files_for_lab/machine.xlsx new file mode 100644 index 0000000..f4ba5e3 Binary files /dev/null and b/files_for_lab/machine.xlsx differ diff --git a/files_for_lab/student_gpa.txt b/files_for_lab/student_gpa.txt index ad44343..455af2d 100644 --- a/files_for_lab/student_gpa.txt +++ b/files_for_lab/student_gpa.txt @@ -1,18 +1,18 @@ -Sophomores Juniors -3.04 2.56 -1.71 2.77 -3.3 2.7 -2.88 3 -2.11 2.98 -2.6 3.47 -2.92 3.26 -3.6 3.2 -2.28 3.19 -2.82 2.65 -3.03 3 -3.13 3.39 -2.86 2.58 -3.49 -3.11 -2.13 -3.27 +Sophomores Juniors +3,04 2,56 +1,71 2,77 +3,3 2,7 +2,88 3 +2,11 2,98 +2,6 3,47 +2,92 3,26 +3,6 3,2 +2,28 3,19 +2,82 2,65 +3,03 3 +3,13 3,39 +2,86 2,58 +3,49 +3,11 +2,13 +3,27 diff --git a/files_for_lab/student_gpa.xlsx b/files_for_lab/student_gpa.xlsx new file mode 100644 index 0000000..30e6f0f Binary files /dev/null and b/files_for_lab/student_gpa.xlsx differ