P-value for pre-test of parallel trends assumption = 0, dynamic aggregation of ATTs not below p-val threshold pre-0 event time

[emmiobara]

Hello,

We're using the did package and the output of the p-value of the Wald statistic for pre-testing the parallel trends assumption = 0 in many of our analyses. However, the dynamic aggregation of the ATTs does not show any of the event times pre-zero as statistically significant. The summary of the group-time ATEs also do not show any of the pre-treatment periods as statistically significant. In addition, all of our plots show the confidence intervals of the pre-zero periods crossing 0. We have checked the coding of the group and time variables and they seem to fit the required data structure.

Any ideas as to why the p-value for the pre-test of parallel trends is 0?

Thanks!

[bcallaway11]

Hi @emmiobara, will you post here or email me a screenshot of the estimated ATT(g,t)'s that you are getting.

Thanks,
Brant

[emmiobara]

Hi @bcallaway11 ,

I've sent you a screenshot of the output via email.

Emmi

[AnneSimon233]

Hi to both of you,

were you able to sort out this issue? I'm experiencing the same issue and would like to be able to get the actual p-values for pre-trends...

Thanks a lot in advance!

Best,
Anne

[emmiobara]

Hi Anne,

I was not able to figure out this issue. Unfortunate, but I'm weirdly glad to see that this isn't just an issue with our data.

Best,
Emmi

[lgoetschius-hilltop]

Hello,

I wanted to follow-up on this issue as well. My team is working on a program evaluation that involves staggered treatment timing and we've seen some results where the P-value for pre-test of parallel trends assumption = 0. Does that mean that there's an error going on somewhere/we should not trust the regression output or is it just a very small p-value?

I'm happy to provide additional information if that would be helpful, thanks!

Best,
Leigh

[AnneSimon233]

Hello everyone,
I'm glad other people are following up on this as well. I would still be happy to hear back about this because I could not fix the issue myself.
Best,
Anne