Skip to content

How does lower Effect Size lead to Lower Investment? #199

Description

@titubs

@JussanN
@ProxyCausal

Hello,
I am not clear how lower effect size ~ lower investment for my dataset. Typically, I would expect the inverse as lower effect size need more user exposure to detect a change, if exist.

Here is my code:


MarketSelections <- GeoLiftMarketSelection(data = df,
                                          treatment_periods = c(180),
                                          N = c(38,39,40), #treatment dmas, out of the total 45 dmas
                                          Y_id = "Y",
                                          location_id = "location",
                                          time_id = "time",
                                          effect_size = seq(0, 0.25, 0.01), 
                                          lookback_window = 1,
                                          include_markets = c(5 distinct markets that I removed from this view),
                                          alpha = 0.1,
                                          Correlations = TRUE,
                                          fixed_effects = TRUE,
                                          side_of_test = "two_sided")

I get this output:

  duration EffectSize Power AvgScaledL2Imbalance Investment   AvgATT
1      180       0.01     1            0.7080668   36,300,423 2451.606
2      180       0.06     1            0.6403676  222,958,296 5072.769
3      180       0.01     1            0.6694795   35,818,012 2603.821
4      180       0.07     1            0.5954214  260,005,746 6063.955
5      180       0.01     1            0.7270462   37,239,591 2931.524
6      180       0.01     1            0.6107752   38,288,706 4028.945
  Average_MDE ProportionTotal_Y abs_lift_in_zero    Holdout rank correlation
1  0.02898100         0.9241685            0.019 0.07583150    1   0.8887812
2  0.05891296         0.9467570            0.001 0.05324301    2   0.9356357
3  0.03043810         0.9116667            0.020 0.08833329    2   0.8979691
4  0.06867882         0.9463459            0.001 0.05365413    4   0.9365457
5  0.03484381         0.9481782            0.025 0.05182180    4   0.8688512
6  0.04712843         0.9750235            0.037 0.02497653    6   0.7508689
  1. As you can see, ID=1 with ES 1% has $36M investment. Whereas ID=2 with ES 6% has $222M investment with markets being the same. I am not sure how to unpack this since I thought higher ES should be more "affordable".

Thanks!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions