01-simple-linear-regression.qmd

---
title: "Simple Linear Regression Lecture Notes"
date: 2024/01/25
---

Below are the lecture notes, based on the slides, for the first Simple Linear Regression lecture. Following these notes are draft lecture notes for subsequent lectures.

{{< include simple-linear-regression-slides-1.qmd >}}

## Outline for Subsequent Lectures in Simple Linear Regression

### Assumptions of Simple Regression

::: incremental
1.  Linear function between x and y
2.  Independence of errors
3.  Normality of errors
4.  Equality of error variances
:::

::: incremental
-   Notice how 3 of 4 focus on errors, NOT the observed variables!
-   These are important for the quality of our model inferences.
:::

### Diagnosing Model Assumptions

#### Linear relationship

##### Good:

```{r}
# label: good-example-linear-relationship

require(tidyverse)

set.seed(2)
linear_data <-
  tibble(
    x = rnorm(30, mean = 100, sd = 15)
    , y = 30 + 10*x + rnorm(30, 0, 30)
  )

linear_data %>%
  ggplot(
    aes(x = x, y = y)
  ) +
  geom_point() +
  theme_bw() +
  ggtitle('Linearly-related Data')
```

------------------------------------------------------------------------

##### Bad:

```{r}
#| label: bad-example-linear-relationship
set.seed(3)
nonlinear_data <-
  tibble(
    x = rnorm(100, mean = 0, sd = 3)
    , y = 30 + 10*x^2 + rnorm(100, 0, 10)
  )

nonlinear_data %>%
  ggplot(
    aes(x = x, y = y)
  ) +
  geom_point() +
  theme_bw() +
  ggtitle('Quadratically-related Data')
```

------------------------------------------------------------------------

#### Independence of Errors

##### Good:

```{r}
#| label: good-example-independent-errors

set.seed(4)
independent_errors_data <-
  tibble(
    x = rnorm(100, mean = 45, sd = 4)
    , y = 5 -10*x + rnorm(100, 0, 6)
    , prediction = 5 - 10*x
    , error = y - prediction
  )

independent_errors_data %>%
  ggplot(
    aes(x = x, y = error)
  ) +
  geom_point() +
  geom_abline(slope = 0, intercept = 0) +
  geom_linerange(aes(ymin = error, ymax = 0)) +
  ggtitle("Residuals vs x-values", "With error distance lines") +
  theme_bw()
```

------------------------------------------------------------------------

##### Bad:

```{r}
#| label: bad-example-independent-errors

set.seed(5)
dependent_errors_data <-
  tibble(
    x = rnorm(100, mean = 2, sd = 4)
    , y = 5 - 10*x + rnorm(100, x, .5)
    , prediction = 5 - 10*x
    , error = y - prediction
  ) 

dependent_errors_data %>%
  ggplot(
    aes(x = x, y = error)
  ) +
  geom_point() +
  geom_abline(slope = 0, intercept = 0) +
  geom_linerange(aes(ymin = error, ymax = 0)) +
  ggtitle("Residuals vs x-values", "With error distance lines") +
  theme_bw()
```

------------------------------------------------------------------------

#### Normality of Errors

##### Good:

```{r}
#| label: good-example-normal-errors
#| layout-ncol: 2
set.seed(6)
normal_errors_data <-
  tibble(
    x = rnorm(1000, mean = 450, sd = 4)
    , y = 50 + 45*x + rnorm(1000, 0, 6)
    , prediction = 50 + 45*x
    , error = y - prediction
  )

normal_errors_data %>%
  ggplot(
    aes(x = x, y = error)
  ) +
  geom_point() +
  geom_abline(slope = 0, intercept = 0) +
  geom_linerange(aes(ymin = error, ymax = 0)) +
  ggtitle("Residuals vs x-values", "With error distance lines") +
  theme_bw()

normal_errors_data %>%
  ggplot(
    aes(x = error)
  ) +
  geom_histogram(bins = 40, fill = 'blue') +
  ggtitle("Distribution of Errors") +
  theme_bw()
```

------------------------------------------------------------------------

##### Bad:

```{r}
#| label: bad-example-normal-errors
#| layout-ncol: 2
set.seed(7)
nonnormal_errors_data <-
  tibble(
    x = rnorm(1000, mean = 450, sd = 4)
    , y = 50 + 45*x + runif(1000, min = -2, max = 2)
    , prediction = 50 + 45*x
    , error = y - prediction
  )

nonnormal_errors_data %>%
  ggplot(
    aes(x = x, y = error)
  ) +
  geom_point() +
  geom_abline(slope = 0, intercept = 0) +
  geom_linerange(aes(ymin = error, ymax = 0)) +
  ggtitle("Residuals vs x-values", "With error distance lines") +
  theme_bw()

nonnormal_errors_data %>%
  ggplot(
    aes(x = error)
  ) +
  geom_histogram(bins = 40, fill = 'red') +
  ggtitle("Distribution of Errors") +
  theme_bw()
```

------------------------------------------------------------------------

#### Equality of Errors

##### Good:

```{r}
#| label: good-example-equal-errors
set.seed(8)
equal_errors_data <-
  tibble(
    x = rnorm(200, mean = -350, sd = 4)
    , y = 50 + 45*x + rnorm(200, 0, 6)
    , prediction = 50 + 45*x
    , error = y - prediction
  )

equal_errors_data <-
  equal_errors_data %>%
  mutate(
    quartile = ntile(x, n = 4)
  )

equal_errors_data %>%
  ggplot(
    aes(x = x, y = error)
  ) +
  geom_point() +
  geom_abline(slope = 0, intercept = 0) +
  geom_linerange(aes(ymin = error, ymax = 0)) +
  ggtitle("Residuals vs x-values", "With error distance lines") +
  theme_bw()

equal_errors_data %>%
  ggplot(
    aes(x = quartile, y = error, group = quartile)
  ) +
  geom_boxplot() +
  ggtitle("Distribution of Errors", "Homogeneous Errors") +
  theme_bw()
```

------------------------------------------------------------------------

##### Bad:

```{r}
#| label: bad-example-equal-errors
set.seed(9)
nonequal_errors_data <-
  tibble(
    x = rnorm(200, mean = -350, sd = 4)
    , y = 50 + 45*x + rnorm(200, (x + 350)^2, 1)
    , prediction = 50 + 45*x
    , error = y - prediction
  )

nonequal_errors_data <-
  nonequal_errors_data %>%
  mutate(
    quartile = ntile(x, n = 4)
  )

nonequal_errors_data %>%
  ggplot(
    aes(x = quartile, y = error, group = quartile)
  ) +
  geom_boxplot() +
  ggtitle("Distribution of Errors", "Heterogenous Errors") +
  theme_bw()
```

### R Diagnostic Plots

#### Model assumptions not violated

```{r}
#| layout-ncol: 2
#| layout-nrow: 2
#| echo: true

plot(lm_height_weight)
```

------------------------------------------------------------------------

#### Model assumptions violated

```{r}
#| layout-ncol: 2
#| layout-nrow: 2
nonequal_errors_lm <-
  lm(y ~ x, data = nonequal_errors_data)

plot(nonequal_errors_lm)
```

### Future Regression Topics

::: incremental
-   No-intercept model
-   Polynomial regression
-   Multiple regression
-   Logistic regression
-   Analysis of Variance (ANOVA)
-   Multivariate regression
:::