Feature Request: Mutually exclusive experiment layers

[jakesciotto]

Is your feature request related to a problem?

Teams that run several experiments at the same time cannot keep the traffic separate. A person can be in two experiments that change the same page or the same flow. The results of both experiments mix, and the team cannot trust the effect of either one.

Today the only tool is a holdout, which removes a percentage of all users from experiments. Issue #33614 asks to target holdouts by earlier experiments, and lists mutual exclusivity as one use case. There is no first-class way to say "these experiments must not share users".

Describe the solution you'd like

Add layers (also called exclusion groups) to experiments:

1. A layer is a named object in a project. It has a traffic split, for example 100% divided into slices.
2. When an experiment is added to a layer, it gets one or more slices. A person is hashed once per layer, and goes to at most one experiment in that layer.
3. Experiments in different layers can overlap. Experiments in the same layer never overlap.
4. The experiment page shows the layer, the slice size, and the other experiments in the layer.
5. The flag evaluation applies the layer slice before the release conditions, in the same place where holdouts are evaluated today.
6. Add a warning when an experiment is not in a layer and another running experiment targets the same flag key prefix, URL, or event.

Describe alternatives you've considered

• Holdout targeting by earlier experiments (Feature Request: Target holdouts by existing experiments / exclude previous experiments #33614). This works for sequential experiments, but not for concurrent ones.
• A cross-experiment exposure view (Cross-experiment user / exposure view #67208). Shows the overlap, but does not prevent it.

Additional context

Related: #33614, #67208. The holdout evaluation in products/experiments is the closest existing mechanism.