notebooks/week4_lecture.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Modules\n",
    "To carry out statistical tests in Python, we will be using an external module called [SciPy](https://www.scipy.org/), and to perform statistical modelling we will use the `ols` function from the external module [statsmodels](https://www.statsmodels.org/stable/index.html). To install these modules, launch the \"Anaconda Prompt (Anaconda3)\" program and run the following commands:\n",
    "\n",
    "`conda install -c anaconda scipy`\n",
    "\n",
    "`conda install -c conda-forge statsmodels`\n",
    "\n",
    "<img src=\"../img/AZ_conda_prompt.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will alias SciPy as `sp`, import the `ols` function from the `statsmodels.formula.api` module, and import numpy, pandas and plotnine using their usual aliases."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import scipy as sp\n",
    "from statsmodels.formula.api import ols\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import plotnine as p9"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data\n",
    "To demonstrate the data analysis functionality of Python, we will use the metabric dataset. Some of the functions we will use do not handle missing data, so we will remove any rows for the dataset where data is missing. As we saw in week 3, we can use the `describe()` method to generate summary statistics for this dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "metabric = pd.read_csv(\"../data/metabric_clinical_and_expression_data.csv\").dropna()\n",
    "metabric.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "metabric.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Statistical tests"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tests for normality\n",
    "When we are deciding which statistical test to use in our analysis, we often need to work out whether the data follows a normal distribution or not, as some tests (e.g. t-test) assume that our data are normally distributed. We can test whether a dataset follows a normal distribution by using the Kolmogorov-Smirnov test. For example, the age at diagnosis looks like it could be normally distributed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "p9.ggplot(metabric, p9.aes(\"Age_at_diagnosis\"))\n",
    "    + p9.geom_histogram()\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To run the Kolmogorov-Smirnov test, we use the `kstest()` function from the scipy stats module:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sp.stats.kstest(metabric[\"Age_at_diagnosis\"], \"norm\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Kolmogorov-Smirnov test has a p value below 0.05, indicating that we can reject the null hypothesis that there is no difference between this distribution and a normal distribution. In other words, the distribution appears non-normal."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In SciPy, the results of most tests are returned as an object. When printed directly to screen this is not very pretty and hard to interpret,  as we can see above. When running the test, we can assign the results object to a variable, and then access the attributes of the results object to print the results in a clearer format:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# run the test and assign the result to a variable\n",
    "age_diagnosis_ks = sp.stats.kstest(metabric[\"Age_at_diagnosis\"], \"norm\")\n",
    "# print the results by retrieving attributes from the result object \n",
    "print(\"Age at diagnosis Kolmogorov-Smirnov test:\")\n",
    "print(\"p value = {}\".format(age_diagnosis_ks.pvalue))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "An alternative to KS test is the Shapiro-Wilk (SW) test. Similar to the KS test. the SW test yield a p-value, but does not compare the distribution at stake with any refrendce distribution.\n",
    "\n",
    "As a rule of thumb, the SW test is preffered over the KS test when the number of dada points is low (usually < 30)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# run the test and assign the result to a variable\n",
    "age_diagnosis_ks = sp.stats.shapiro(metabric[\"Age_at_diagnosis\"])\n",
    "# print the results by retrieving attributes from the result object \n",
    "print(\"Age at diagnosis Shapiro-Wilk:\")\n",
    "print(\"p value = {}\".format(age_diagnosis_ks[1]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Correlation\n",
    "We often want to test whether two continuous variables are related to each other, and we can do this by calculating a correlation. For example, there appears to be a relationship between the expression of the ESR1 gene and the GATA3 gene:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "p9.ggplot(metabric, p9.aes(\"ESR1\", \"GATA3\"))\n",
    "    + p9.geom_point()\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For normally distributed data, we can calculate the Pearson's correlation using the `pearsonr()` function. `pearsonr()` returns the results as a tuple rather than an object, so we need to access the coefficient and p value using indexing:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ESR1_GATA3_pearson = sp.stats.pearsonr(metabric[\"ESR1\"], metabric[\"GATA3\"])\n",
    "print(\"Pearson correlation between ESR1 & GATA3:\")\n",
    "print(\"coefficient = {}\".format(ESR1_GATA3_pearson[0]))\n",
    "print(\"p value = {}\".format(ESR1_GATA3_pearson[1]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For data that is not normally distributed, we can calculate the Spearman rank correlation. For example, a scatter plot of tumour size versus mutation count suggests that these are not normally distributed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "p9.ggplot(metabric, p9.aes(\"Tumour_size\", \"Mutation_count\"))\n",
    "    + p9.geom_point()\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can calculate the Spearman rank correlation using the `spearmanr()` function, again accessing the results using indexing:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "size_mutation_spearman = sp.stats.spearmanr(metabric[\"Tumour_size\"], metabric[\"Mutation_count\"])\n",
    "print(\"Spearman rank correlation between tumour size and mutation count:\")\n",
    "print(\"coefficient = {}\".format(size_mutation_spearman[0]))\n",
    "print(\"p value = {}\".format(size_mutation_spearman[1]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## T-test\n",
    "To test whether the mean value of a continuous variable is significantly different between two different groups, we can use the t-test for normally distributed data. For example, age at diagnosis appears to be lower for ER-negative tumours compared with ER-positive tumours:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "p9.ggplot(metabric, p9.aes(\"ER_status\", \"Age_at_diagnosis\"))\n",
    "    + p9.geom_violin()\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can use the `ttest_ind()` function to carry out the t-test, which confirms that we can reject the null hypothesis that age at diagnosis is not different between ER positive and negative tumours. Note that `ttest_ind()` takes two arguments, which are the values of the two groups. Rather than extracting these values and assigning them to separate variables, we can do the data extraction within the function call:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ER_age_t = sp.stats.ttest_ind(\n",
    "    # select samples with Negative ER_status and extract the Age_at_diagnosis values\n",
    "    metabric[metabric[\"ER_status\"]==\"Negative\"][\"Age_at_diagnosis\"],\n",
    "    # select samples with Positive ER_status and extract the Age_at_diagnosis values\n",
    "    metabric[metabric[\"ER_status\"]==\"Positive\"][\"Age_at_diagnosis\"]\n",
    ")\n",
    "print(\"t test of age at diagnosis for ER_status Negative vs Positive:\")\n",
    "print(\"t = {}\".format(ER_age_t.statistic))\n",
    "print(\"p = {}\".format(ER_age_t.pvalue))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we have data that is not normally distributed we may want to use the Mann-Whitney U test, also known as the Wilcoxon rank-sum test, which is the non-parametric equivalent of the t-test. For example, survival time does not follow a normal distribution, but it still appears to be different between ER positive and ER negative tumours:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "p9.ggplot(metabric, p9.aes(\"ER_status\", \"Survival_time\"))\n",
    "    + p9.geom_violin()\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can use the `mannwhitneyu()` function to run the Mann-Whitney U test, which confirms that we can reject the null hypothesis that age at diagnosis is not different between ER positive and negative tumours. Again, we are subsetting and selecting the data within the function call:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ER_survival_MWU = sp.stats.mannwhitneyu(\n",
    "    # select samples with Negative ER_status and extract the Age_at_diagnosis values\n",
    "    metabric[metabric[\"ER_status\"]==\"Negative\"][\"Age_at_diagnosis\"],\n",
    "    # select samples with Positive ER_status and extract the Age_at_diagnosis values\n",
    "    metabric[metabric[\"ER_status\"]==\"Positive\"][\"Age_at_diagnosis\"]\n",
    ")\n",
    "print(\"Mann-Whitney U test of survival time for ER_status Negative vs Positive:\")\n",
    "print(\"f = {}\".format(ER_survival_MWU.statistic))\n",
    "print(\"p = {}\".format(ER_age_t.pvalue))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ANOVA\n",
    "If we want to test for a difference in the mean value of a continuous variable between >2 groups simultaneously, we can use the analysis of variance (ANOVA). For example, we may want to test for differences between survival times between different cancer types, which appear to be different:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "p9.ggplot(metabric, p9.aes(\"Cancer_type\", \"Survival_time\"))\n",
    "    + p9.geom_boxplot()\n",
    "    + p9.theme(axis_text_x = p9.element_text(angle=45, hjust=1))\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can use the `f_oneway()` function to run ANOVA, which shows that we cannot reject the null hypothesis that there is no difference in survival time between cancer types:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type_survival_anova = sp.stats.f_oneway(\n",
    "    # select samples with Breast cancer and extract the Survival_time values\n",
    "    metabric[metabric[\"Cancer_type\"]==\"Breast\"][\"Survival_time\"],\n",
    "    # select samples with Breast cancer and extract the Survival_time values\n",
    "    metabric[metabric[\"Cancer_type\"]==\"Breast Invasive Ductal Carcinoma\"][\"Survival_time\"],\n",
    "    # select samples with Breast cancer and extract the Survival_time values\n",
    "    metabric[metabric[\"Cancer_type\"]==\"Breast Invasive Lobular Carcinoma\"][\"Survival_time\"]\n",
    ")\n",
    "print(\"ANOVA of survival time for different cancer types:\")\n",
    "print(\"f = {}\".format(type_survival_anova.statistic))\n",
    "print(\"p = {}\".format(type_survival_anova.pvalue))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chi-square test\n",
    "If we have two categorical variables of interest, and we want to test whether the status of one variable is linked to the status of the other, we can use the chi-square test. For example, we may want to test whether the ER status of a tumour (Positive or Negative) is linked to the PR status (Positive or Negative). First, we need to format the data into a contingency table, containing counts of positive and negative values for ER and PR:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# use the crosstab function to make a contingency table of the total numbers of patients that are ER+ & PR+, ER+ & PR-, ER- & PR+, and ER- & PR-\n",
    "ER_PR_contingency = pd.crosstab(metabric[\"ER_status\"], metabric[\"PR_status\"])\n",
    "ER_PR_contingency"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we use the `chi2_contingency()` function to run the chi-square test, and assign the results to a variable. This shows that we can reject the null hypothesis that ER and PR status are independent. The results are returned as a tuple rather than an object, so we retrieve them by using indexing:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ER_PR_chi = sp.stats.chi2_contingency(ER_PR_contingency)\n",
    "print(\"Chi-square test for ER and PR status:\")\n",
    "print(\"Chi-square value = {}\".format(ER_PR_chi[0]))\n",
    "print(\"p value = {}\".format(ER_PR_chi[1]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Transformation\n",
    "When working with large datasets, we often have variables with very different ranges and distributions of values. For some analyses, particularly statistical modelling, it is helpful to be able to apply a mathematical transformation to a set of values, which rescales the values and makes their distribution and range more similar to other variables in the dataset. For example, in the Metabric dataset the distribution of tumour sizes is highly left-skewed, as most tumours are small but a few are very large:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "p9.ggplot(metabric, p9.aes(\"Tumour_size\"))\n",
    "    + p9.geom_histogram()\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To perform transformations on this data, we can use some functions from numpy:\n",
    "- `sqrt()` = square-root transform\n",
    "- `log2()` = log-transform with base 2\n",
    "- `log10()` = log-transform with base 10\n",
    "\n",
    "All of these functions return a pandas series of transformed values. To retain the original (untransformed) data, we can add these transformed values to the metabric dataframe as a new column:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "metabric[\"Tumour_size_sqrt\"] = np.sqrt(metabric[\"Tumour_size\"])\n",
    "metabric[\"Tumour_size_log2\"] = np.log2(metabric[\"Tumour_size\"])\n",
    "metabric[\"Tumour_size_log10\"] = np.log10(metabric[\"Tumour_size\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After transformation, the tumour sizes look closer to being normally distributed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# select the variables of interest\n",
    "tumour_size_tranformations = metabric.loc[:,[\"Patient_ID\", \"Tumour_size\", \"Tumour_size_sqrt\", \"Tumour_size_log2\", \"Tumour_size_log10\"]]\n",
    "# rename the columns for ease of plotting\n",
    "tumour_size_tranformations.columns = [\"Patient_ID\", \"Untransformed\", \"sqrt\", \"log2\", \"log10\"]\n",
    "# reformat the untransformed and transformed data into three columns ahead of plotting\n",
    "tumour_size_tranformations = pd.melt(tumour_size_tranformations, id_vars=\"Patient_ID\", var_name=\"Transformation\", value_name=\"Size\")\n",
    "# plot faceted histogram\n",
    "(\n",
    "p9.ggplot(tumour_size_tranformations, p9.aes(\"Size\"))\n",
    "    + p9.facet_wrap(\"~Transformation\", nrow=2, scales=\"free\")\n",
    "    + p9.geom_histogram(bins=50)\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Modelling"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Simple linear regression\n",
    "If we have a continuous variable, and we want to model its relationship with another variable, we can use simple linear regression. In linear regression we call the variable of interest the **response**, and the other variable the **predictor**. The mathematical details of linear regression are beyond the scope of this course, but in the case of simple linear regression it basically amounts to fitting a line through the data that is closest to all of the points. For example, we may want to predict survival time based on tumour size, because survival time appears to differ across the range of tumour sizes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "p9.ggplot(metabric, p9.aes(\"Tumour_size\", \"Survival_time\"))\n",
    "    + p9.geom_point()\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In Python, we can run simple linear regression using the `ols` function from the `statsmodels` package. There are three steps to completing this analysis:\n",
    "1. **Instantiate** the model: create an object that holds the model specification and the input dataset. In the model specification, the response is to the left of the tilda `~` and the predictor is to the right\n",
    "2. **Fit** the model: fit the specified model to the data using the `fit()` function, and assign the results object to a variable\n",
    "3. **Display** the results: use the `summary()` method of the results object to return a detailed breakdown of the model characteristics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# instantiate model\n",
    "simple_model = ols(\"Survival_time~Tumour_size\", data=metabric)\n",
    "# fit the model\n",
    "simple_results = simple_model.fit()\n",
    "# display the results\n",
    "simple_results.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The model summary contains a lot of detailed information, but we can create a more concise report of the results by extracting the results of interest e.g. the r2 value, the F-statistic and its p value:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Simple linear regression: Survival_time~Tumour_size\")\n",
    "print(\"r2 = {}\".format(simple_results.rsquared))\n",
    "print(\"F-statistic = {}\".format(simple_results.fvalue))\n",
    "print(\"F-statistic p value= {}\".format(simple_results.f_pvalue))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After fitting a linear regression model, we usually want to carry out some basic checks of the model characteristics. This is because linear regression makes some assumptions about the data and our model, and if the data that we have fitted our model to has violated these assumptions, then the predictions from the model may not be reliable. We will not cover these checks in this session as they are beyond the scope of the course, but if you want information on how to do this then please see the [statsmodels documentation](https://www.statsmodels.org/stable/index.html)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we are happy with the checks of model characteristics, we can use the model to predict what the value of our response variable will be, given a certain value for the predictor variable. We do this using the `predict()` method of the results object, which takes the value of the predictor variable as the argument:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "simple_results.predict({\"Tumour_size\": 125})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our model predicts a survival time of 8.6 for a tumour size of 125; however, the low r2 value for this model (r2=0.053) indicates that it fits the data very poorly, so we may not be very confident in this prediction."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Multivariate linear regression\n",
    "When we are analysing more complex processes, we often need to consider the influence of multiple predictors simultaneously. One way to do this is by using multivariate linear regression, which models the relationship between the response and two or more predictors. For example, we may wish to model the effect on survival time of tumour size, tumour stage, cancer type and ER status. To do this we repeat the simple regression process described above, but specify multiple predictors when instantiating the model. When viewing the results, we extract the `pvalues` attribute of the results object to print the p values associated with each predictor:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# instantiate model\n",
    "complex_model = ols(\"Survival_time~Tumour_size + Tumour_stage + Cancer_type + ER_status\", data=metabric)\n",
    "# fit the model\n",
    "complex_results = complex_model.fit()\n",
    "# print the results of interest\n",
    "print(\"Complex linear regression: Survival_time~Tumour_size + Tumour_stage + Cancer_type + ER_status\")\n",
    "print(\"r2 = {}\".format(complex_results.rsquared))\n",
    "print(\"F-statistic = {}\".format(complex_results.fvalue))\n",
    "print(\"F-statistic p value= {}\".format(complex_results.f_pvalue))\n",
    "print(\"p values for each predictor:\")\n",
    "print(complex_results.pvalues)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Including these extra predictors has almost doubled the r2, but the model fit is still quite poor (r2=0.098). Given the complexity of breast cancer biology and the relative simplicity of our analysis, this isn't a big surprise!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercises"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 1\n",
    "Is there a significant difference between the tumour size of patients who received chemotherapy versus patients that did not receive chemotherapy? Use either the t-test or Mann-Whitney U test, and provide a visualization to compare tumour size between patients who did or did not receive chemotherapy. When deciding which test to use, remember to check whether the data is normally distributed or not."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 2\n",
    "Is there a correlation between tumour size and survival time? If so, does the correlation become weaker or stronger after tumour size is log10-transformed? Generate a visualization of the relationship between log10-transformed tumour size and survival time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 3\n",
    "Make a contingency table of the number of patients that did or did not receive chemotherapy and did or did not receive radiotherapy, and use a chi-square test to investigate whether the incidence of these treatments are independent of each other."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}