tidy for AGGTEobj objects and confidence intervals

[tkeskinturk]

Hello!

I am using the aggte function to aggregate a model, and I would like to grab the output through tidy.

That said, there seems to be a mismatch between the tidy output and actual results: the confidence intervals coming in tibbles from tidy are different than the confidence intervals from aggte.

There seems to be no issue for the estimates and SEs (and using the 1.96 cutoff produces the same CIs as that of summary).

Here is a reproducible example. I am assuming that there is a bug somewhere that might cause problems.

Big fan of the package!

library(tidyverse)
library(did)

d <- mpdta
m <- att_gt(
  yname = 'lemp',
  tname = 'year',
  idname = 'countyreal',
  gname = 'first.treat',
  data = d,
  allow_unbalanced_panel = TRUE,
  control_group = "notyettreated"
)

m_agg <- aggte(m, type = "simple")

summary(m_agg)
#> 
#> Call:
#> aggte(MP = m, type = "simple")
#> 
#> Reference: Callaway, Brantly and Pedro H.C. Sant'Anna.  "Difference-in-Differences with Multiple Time Periods." Journal of Econometrics, Vol. 225, No. 2, pp. 200-230, 2021. <https://doi.org/10.1016/j.jeconom.2020.12.001>, <https://arxiv.org/abs/1803.09015> 
#> 
#> 
#>      ATT    Std. Error     [ 95%  Conf. Int.]  
#>  -0.0398        0.0123    -0.0638     -0.0158 *
#> 
#> 
#> ---
#> Signif. codes: `*' confidence band does not cover 0
#> 
#> Control Group:  Not Yet Treated,  Anticipation Periods:  0
#> Estimation Method:  Doubly Robust
tidy(m_agg)
#>     type    estimate  std.error    conf.low  conf.high point.conf.low
#> 1 simple -0.03976363 0.01225156 -0.01176106 0.03626419    -0.01176106
#>   point.conf.high
#> 1      0.03626419

^{Created on 2024-02-07 with reprex v2.1.0}

[MatthieuStigler]

I confirm! The problem is that the current CRAN version as of August 2024 (2.1.2) contains a mistake in did:::tidy.AGGTEobj:

conf.high = x$overall.se + stats::qnorm(1 - x$DIDparams$alp/2) * x$overall.se

This seems to have been corrected in this fix e28dd9e on Aug 30, 2023, but unfortunately, the fixed version has not been pushed to CRAN.

So you might want to get the github version devtools::install_github("bcallaway11/did"), which gives you the correct result, see below.

@bcallaway11 do you have plans to push to CRAN the current dev version soon? As the current CRAN version is two years old and contains critical bugs such as this one, this would be greatly appreciated! Thanks a lot :-)

library(did)
packageVersion("did")
#> [1] '2.1.2'

## run standard
out_all <- att_gt(
  yname = 'lemp',
  tname = 'year',
  idname = 'countyreal',
  gname = 'first.treat',
  data = mpdta,
  allow_unbalanced_panel = TRUE,
  control_group = "notyettreated")

out_agg <- aggte(out_all, type = "simple", bstrap = FALSE)

out_agg ## was showing -0.0635     -0.0164
#> 
#> Call:
#> aggte(MP = out_all, type = "simple", bstrap = FALSE)
#> 
#> Reference: Callaway, Brantly and Pedro H.C. Sant'Anna.  "Difference-in-Differences with Multiple Time Periods." Journal of Econometrics, Vol. 225, No. 2, pp. 200-230, 2021. <https://doi.org/10.1016/j.jeconom.2020.12.001>, <https://arxiv.org/abs/1803.09015> 
#> 
#> 
#>      ATT    Std. Error     [ 95%  Conf. Int.]  
#>  -0.0398        0.0121    -0.0634     -0.0161 *
#> 
#> 
#> ---
#> Signif. codes: `*' confidence band does not cover 0
#> 
#> Control Group:  Not Yet Treated,  Anticipation Periods:  0
#> Estimation Method:  Doubly Robust
tidy(out_agg) ## was showing -0.01155222 0.03562024
#>     type    estimate  std.error    conf.low   conf.high point.conf.low
#> 1 simple -0.03976363 0.01205242 -0.06338594 -0.01614131    -0.06338594
#>   point.conf.high
#> 1     -0.01614131

^{Created on 2024-08-20 with reprex v2.1.1}