Make `revision_analysis` reflect less common but problematic revisions

[brookslogan]

Our current metrics don't seem to reflect the frustrations we've had with revisions in certain data sources, e.g., NHSN. Consider adding:

• Key quantiles of signed percent error by lag, filtered/adjusted to not overreact to low denominators
• Something to reflect presence of major rewrites changing the scale of the data set for an epigroup

[brookslogan]

WIP

nhsn_flu_archive %>%
  filter(between(time_value, as.Date("2024-10-01"), as.Date("2025-05-31"))) %>%
  {
    archive = .
    ekvls <- archive$DT %>% as.data.frame() %>% as_tibble() %>% expand(crossing(nesting(geo_value), version, lag = 7*c(1,2,3,4)))
    get_lagged_signal_vectorized(archive, ekvls["geo_value"], ekvls[["version"]], ekvls[["lag"]], "value", ltcf_limit = 0) %>%
      rename(version = date_to_match) %>%
      # mutate(time_value = epiprocess:::time_minus_n_steps(version, lag, archive$time_type))
      mutate(time_value = version - lag)
  } %>%
  left_join(nhsn_flu_archive %>% epix_as_of(.$versions_end), by = c("geo_value", "time_value")) %>%
  # mutate(include = !is.na(value.x) & !is.na(value.y) & value.y >= 1) %>%
  mutate(include = !is.na(value.x) & !is.na(value.y) & value.y >= 10) %>%
  # mutate(include = !is.na(value.x) & !is.na(value.y) & value.y >= 30) %>%
  mutate(include = include & ! geo_value %in% c("az", "wa")) %>%
  summarize(.by = lag,
            MAPE = 100*mean(abs((value.x - value.y)/value.y)[include]),
            MdAPE = 100*median(abs((value.x - value.y)/value.y)[include]),
            highAPE = 100*quantile(abs((value.x - value.y)/value.y)[include], probs = 0.9),
            whighAPE = 100*quantile(abs((value.x - value.y)/value.y)[include], weights = value.y[include], probs = 0.9),
            whighSPE = 100*quantile(((value.x - value.y)/value.y)[include], weights = value.y[include], probs = 0.9),
            wlowSPE = 100*quantile(((value.x - value.y)/value.y)[include], weights = value.y[include], probs = 0.1),
            whighSPE2 = 100*quantile(((value.x - value.y)/value.y)[include], weights = value.y[include], probs = 0.95),
            wlowSPE2 = 100*quantile(((value.x - value.y)/value.y)[include], weights = value.y[include], probs = 0.05),
            mean(include),
            sum(include)) %>%
  {
    print(.)
  } %>%
  # ggplot(aes(lag, MdAPE)) %>%
  # ggplot(aes(lag, MAPE)) %>%
  # ggplot(aes(lag, whighAPE)) %>%
  # ggplot(aes(lag, whighSPE)) %>%
  # ggplot(aes(lag, wlowSPE)) %>%
  # ggplot(aes(lag, whighSPE2)) %>%
  # ggplot(aes(lag, wlowSPE2)) %>%
  # `+`(geom_line())
  {}

[dsweber2]

I think part of what distinguishes the large jumping revisions in nhsn is that they revise every time_value on a particular revision, which is a way of slicing the data that isn't accounted for in the function at all