model-evaluation.qmd

---
title: "Forecast Analysis"
format:
  html:
    embed-resources: true
editor: visual
---

## Problem set

You will be using your model_id, climatology and persistence in the analysis below. You will be using site_ids BARC and SUGG (since they have year-around data)

Resources: 

<https://frec-5174.github.io/eco4cast-in-R-book/visualizing.html>

<https://frec-5174.github.io/eco4cast-in-R-book/process-model-forecast-evaluation.html>

Find your forecast scores here:

<https://radiantearth.github.io/stac-browser/#/external/raw.githubusercontent.com/eco4cast/neon4cast-ci/main/catalog/scores/collection.json>

climatalogy and persistenceRW are the null model_id

### Question 1

Plot climatology, persistence, and your model for a single forecast day (one reference_datetime) on the same plot.  Use geom_ribbon to plot the uncertainty, geom_line to plot the median, and geom_point to plot the observations.

```{r}


```

### Question 2

Based on visual inspection of your plot, how do the median of model differ in how they represent the observations.

### Question 3

Based on visual inspection of your plot, how does the uncertainty of each model forecasts differ in capacity to represent the observations.

### Question 4

Calculate the mean CRPS for the three models(averaged across all horizons, sites, and reference_datetimes). Which model has the lower score?

```{r}


```

### Question 5

Plot the mean CRPS vs horizon for all three models.  How does the performance change as you forecast further in the future?

```{r}


```

### Question 6

Plot the mean CRPS separately for each site_id for all three models.  How does performance differ between sites?

```{r}


```

### Question 7

Which forecasting best practices are addressed with your forecasts and the analysis above?  See <https://frec-5174.github.io/eco4cast-in-R-book/best-practices.html>.